Refactor how users and login are used in test scenarios

[charleslbryant]

We have found that having to define the user for each test scenario (e.g. user, password) is a source of a lot of duplication. We almost had a situation that would have forced updating user passwords in hundreds of tests... no good.

We should refactor the concept of a user into a separate data file. Then we can point tests to a particular user. We need to make sure that we distinguish between users that won't be updated in test from those that will be updated in a test. If a user is updated in a test it should only be used in one test to reduce flakes.

I envision this as a possible structure:

<{ "Users": [ { "Id": "Primary User", "Scope": "AnyTest", "Data": { "Username": "tester1@abc.comx", "Password": "Abcd1234" } }, { "Id": "Invalid User", "Scope": "AnyTest", "Data": { "Username": "invalid1@abc.comx", "Password": "Abcd1234" } }, { "Id": "Profile Test", "Scope": "SingleTest", "Data": { "Username": "profile1@abc.comx", "Password": "Abcd1234" } } ] }>

Then we can consume this user in test data object by indicating the User Id to use in the test.

We would also need an easy way to hide this so that no changes to tests are necessary and both the current way of getting users and this new way work.

We should also look into some way of abstracting the need to worry about login in tests at all. We could use a tag or other mechanism to identify test scenarios that require a logged in use and do the log in in the background.

If this works, it may be useful to extend the concept to any data that is repeated in multiple test scenarios. Maybe an extension of Suite, Feature, and Scenario that adds data that can be pulled from each of these scopes. This may even negate the need to define the scope for users (e.g. AnyTest, SingleTest) since we can define the AnyTest users at the proper scope and the single test users can be defined in the scenario data as we do today.