Today:
- More automated capturing of screen shots
- Started adding Automation Ids to various UI elements in Cantabile to make it easier to locate in invoke different actions.
- Lots of tweaks to the scripts and utility library I’m using to make it easier and more programmatic (ie: more “do exactly this” and less “send these keys”)
- Now capturing nearly 60 screen shots of different UI screens.
Tedious work, but I’m going to persevere with trying getting screenshots of everything in Cantabile.
About UIAutomation
I’ve mentioned UIAutomation a few times now and thought I’d provide a little explainer for those who don’t know what this is.
In GuiKit the entire UI is manually drawn to a big bitmap (image) and then copied to the screen. This allows for nice composition of various views and elements and a very flexible UI framework.
The downside is the content of that bitmap is opaque to Windows and to other programs. Without additional support, screen readers don’t work and utilities programs like AutoHotKey, FlaUI, WinAppDriver have no visibility as to what’s inside a particular window.
That’s where UIAutomation comes in… it’s a separate interface into a window that lets other programs find out about things inside it. eg: a screen reader can ask “what’s at this point”, to which the UIAutomation interface might say “a button” to which the screen reader can ask “what’s the text on the button” to which the app might return “OK”… and the screen reader can highlight the button and speak the word “OK” so a vision impaired user can hear what’s on the screen.
That’s the basic idea, but the UIAutomation interfaces also provide the ability to programatically control a program. eg: instead of a screen reader interacting with the app, perhaps a script program might run a set of actions like… find the File menu, click it, find the Open command, click it, type “MySong”, find the OK button, click it.
So that’s what I’m doing with this automated UI testing - writing script programs that run externally to Cantabile, but drive it to do various actions and then take screenshots of the relevant parts of the screen.
As an example, here’s a script which captures the audio port dialog boxes:
// Use the Edit Find command to find the mda plugin
_fixture.InvokeCommand("edit.globalFind");
var w = _fixture.Session.FindWindow(x => x.Name == "Find");
w.FindById("textSearch").Value = "mda";
w.FindById("buttonOK").Click();
// Open audio ports window
_fixture.InvokeCommand("edit.audioPorts");
var window = _fixture.Session.FindWindow(x => x.Name.StartsWith("Edit Ports - "));
window.CaptureScreenshot("AudioPorts");
// Get all the list items
var list = window.FindById("listPortsAndChannels");
var items = list.FindAll(x => x.ControlType == ControlType.ListItem);
// Select the port, edit and capture
items[0].Select();
window.FindById("buttonEdit").Invoke();
var wPortName = _fixture.Session.FindWindow(x => x.Name.StartsWith("Audio ") && x.Name.EndsWith(" Port"));
wPortName.CaptureScreenshot("AudioPortName");
wPortName.Close();
// Select channel, edit and capture
items[1].Select();
window.FindById("buttonEdit").Invoke();
var wPort = _fixture.Session.FindWindow(x => x.Name.StartsWith("Edit ") && x.Name.EndsWith(" Channel"));
wPort.CaptureScreenshot("AudioPort");
wPort.Close();
// Close audio ports dialog
window.Close();
And here’s what it captured:
Brad
