You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: uitests/README.md
+15-12Lines changed: 15 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -72,20 +72,21 @@ Here are the steps involved in running the tests:
72
72
73
73
* VSC UI needs be a top level window for elements to receive focus. Hence when running tests, try not do anything else.
74
74
* For each test we create a whole new folder and open that in VS Code:
75
-
- We could use `git reset`, however on Windows, this is flaky if VSC is open.
76
-
- Deleting files on `Windows` is flaky due to files being in use, etc.
77
-
- Majority of the issues are around `fs` on `windows`
78
-
- The easies fix for all of this is simple
79
-
- create new folders for every test.
75
+
-We could use `git reset`, however on Windows, this is flaky if VSC is open.
76
+
-Deleting files on `Windows` is flaky due to files being in use, etc.
77
+
-Majority of the issues are around `fs` on `windows`
78
+
-The easies fix for all of this is simple
79
+
-create new folders for every test.
80
80
*`chromedriver` only supports arguments that begin with `--`. Hence arguments passed to VSC are limited to those that start with `--`.
81
81
*`Terminal` output cannot be retrieved using the `driver`. Hence output from terminal cannot be inspected.
82
-
- Perhaps thi sis possible, but at the time of writinng this I couldn't find a solution.
83
-
- I believe the `Terminal` in VSC is `SVG` based, hence reading text is out of the question.
82
+
- Perhaps thi sis possible, but at the time of writinng this I couldn't find a solution.
83
+
- I believe the `Terminal` in VSC is `SVG` based, hence reading text is out of the question.
84
+
- (This is made possible by writing the command to be executed into `commands.txt`, and letting the bootstrap extension read that file and run the command in the terminal using the VSC API).
84
85
* Sending characters to an input is slow, the `selenium` send text one character at a time. Hence tests are slow.
85
86
* Sending text to an editor can be flaky.
86
-
- Assume we would like to `type` some code into a VSC editor.
87
-
- As `selenium` sends a character at a time, VSC kicks in and attempts to format/autocomplete code and the like. This interferes with the code being typed out.
88
-
- Solution: Copy code into clipboard, then pase into editor.
87
+
-Assume we would like to `type` some code into a VSC editor.
88
+
-As `selenium` sends a character at a time, VSC kicks in and attempts to format/autocomplete code and the like. This interferes with the code being typed out.
89
+
-Solution: Copy code into clipboard, then pase into editor.
89
90
*`Behave` does not generate any HTML reports
90
91
- Solution, we generate `cucumber` compliant `json` report. Hence the custom formatter in `report.py`.
91
92
- Using a `cucumber json` report format allows us to use existing tools to generate other HTML reports out of the raw `json` files.
@@ -101,6 +102,7 @@ Here are the steps involved in running the tests:
101
102
- An even better way, is to use the VSC api to update the settings (via the bootstrap API) or edit the settings file directly through the UI.
102
103
- Updating settings through the editor (by editing the `settings.json` file directly is not easy, as its not easy to update/remove settings).
103
104
- Using the API we can easily determine when VSC is aware of the changes (basically when API completes, VSC is aware of the new settings).
105
+
- (This is made possible by writing the settings to be updated into `settingsToUpdate.txt`, and letting the bootstrap extension read that file and update the VSC settings using the VSC API).
104
106
105
107
## Files & Folders
106
108
@@ -125,7 +127,7 @@ Here are the steps involved in running the tests:
125
127
126
128
* For more details please check `build/ci`.
127
129
* We generally try to run all tests against all permutations of OS + Python Version + VSC
128
-
- I.e. we run tests across permutations of the follows:
130
+
-I.e. we run tests across permutations of the follows:
129
131
- OS: Windows, Mac, Linux
130
132
- Python: 2.7, 3.5, 3.6, 3.7
131
133
- VSC: Stable, Insiders
@@ -146,7 +148,8 @@ Here are the steps involved in running the tests:
146
148
## Miscellaneous
147
149
148
150
* Use the debug configuration `Behave Smoke Tests` for debugging.
149
-
* In order to pass custom arguments to `Behave`, refer to the `CLI` (pass `behave` specific args after `--` in `python uitests test`)
151
+
* In order to pass custom arguments to `Behave`, refer to the `CLI` (pass `behave` specific args after `--` in `python uitests test`).
152
+
- E.g. `python uitests test -- --tags=@wip --more-behave-args`
150
153
* Remember, the automated UI interactions can be faster than normal user interactions.
151
154
- E.g. just because we started debugging (using command `Debug: Start Debugging`), that doesn't mean the debug panel will open immediately. User interactions are slower compared to code execution.
152
155
- Solution, always wait for the UI elements to be available/active. E.g. when you open a file, check whether the corresponding elements are visible.
0 commit comments