Detailed Notes
Welcome, if you are a tester or developer interested in mobile application automation testing or are new to test automation tools than you defiantly need to check out TestProject. However, if you are not familiar with this tool that's OK. In Mobile Testing With Test Project we cover all the basics necessary to start mobile testing in Test Project 2.0.
TestProject is a community driven tool and has a free forever plan that is fully featured and that you can get started with in moments. In addition they have a lot of documentation that will help you with all the details you need to use TestProject to create effective test automation that your whole team can use.
Before you begin you will need to:
First, go back to the previous video on Test Project A Review of Version 2.0 and watch the section on setup. In addition, you will need to start the Test Project Agent so that Test Project can record against you local simulators and mobile devices. However, depending on your mobile device will determine what you will need to install and set up next. At this point, you will need Android Studio for Android development, Xcode / iOS Simulator for iOS, or both if you are going to be testing on both types of devices. Finally, if you plan on running your mobile tests out side of test project in a then you will need to export your tests to code. As a result you will need to have an IDE like Eclipse and run them using Appium
Transcript Text
[Music] all right welcome today we are going to be covering the second part of test project which is using test project for mobile testing we are going to look at how to set up a mobile device for testing look at how to create a test project mobile test we're going to record a mobile test with a simulator and a device and we're going to execute and run the mobile test there are some tools you'll need to get started if you are going to use test project you need to install the test project agent so if you haven't done that already go back to the previous video on test project and follow the steps to install the agent you will also need android studio eclipse some maven dependencies and xcode and an ios simulator if you're on a mac in java when we export our mobile test or any of our test project tests to code we're going to need some java maven dependencies in order to use these or run this code from java or from eclipse we'll need to make sure we include selenium selenium java we need to get io java client and i o test project for mobile testing when you export your mobile tests they export to appium so you'll need the appium client we have not discussed appium in any of our classes to date so i'm going to keep this very high level when i discuss appium but i will be doing a future presentation on how to set up and run mobile testing with that view in the future so let's get started the slides for now first if you're on a mac you need to do a few things to get started so the first thing you want to do is you want to open up xcode so once xcode loads you can go under the xcode option and go to where is it here it is so xcode open developer tools you want to open up the simulator app now the first time you open this you want to bookmark it you want to make sure you either pin it add it to create a shortcut to it or something because really you don't want xcode open because it's a good memory drain on your device but you do need the ios simulator and this can run separate from xcode the other thing you want to do is you're going to want to install if you're doing android development you want to make sure that you install android studios and once you install android studios and you get it running you need to come down here to the more action section if you click that you want to go to the avd manager and you need to actually start one of your virtual devices for android now if you have a device kicking around for android you can go ahead and plug it into your machine and test project will recognize your device as long as it's a fairly recent device i found out with the last update that all my devices are so archaic they won't load because i can't load the latest android operating system so unfortunately running this on device i won't be able to do today but i will show you through emulators the other caveat for running it with device if you are trying to use ios and you want to do this on windows you will need an apple developer license so you will need to register to get apple developer id so that you can actually interact with your device real time through test project so that's the setup so if you do those first and get your emulators running it's going to make your life a whole lot easier once you get into test project so now we open up our browser and we log into test project so once we log in it takes us to our project page where we have our previous examples for doctor who and my first test from last presentation and i have two new tests in here i have one for ios and one for android so i'm going to put this to the right here so let's run these so let's run the ios so when i click run it asks me okay which project do i want to run this one i called it my first agent and i currently have iphone 13 pro running so i'll slide down to the bottom here and for some reason and i don't know why test project likes to put all your running emulators or your devices at the bottom of the list so you have to scroll all the way to the bottom to find the one that you want however once you find the devices you can click this and it will show you only the ones that you want to see all right so we'll go down we'll take this guy click next one thing that is recommended is that you always override the default input parameters unless you have a data source and for this simple test i'm just going to override the input parameters and run with what's in my test i click run and while that one's running i'm going to click run for android because you can run these in parallel and as you can see it sees our android device so i will select this one as well and click next and i will again use the defaults so while these are running i'm going to switch over to our monitor so we can see what's going on so here's our monitor our tests are preparing so iphone just reset so it's reset back to default and so did android and as you can see on the android emulator it was actually it's going to run our default app that we were running in react native however apple because i don't have my apple id it's expired i couldn't deploy our sdk or the api to iphone so instead we're just running a test testing the safari app going through and trying to purchase an ipad pro so what you're seeing there on the iphone is i did a couple slides sliding back down and then i'm going to select the 11 inch display and then i selected gray and the test completed so 100 it was now that it's finished it's going to shut down the agent up here is our little monitor this will tell us what is running so we have no workers running at the moment so just finished and it's done so one of the reasons you want to make sure that you reset your device every time you run is when you're running with emulation one of the things i've noticed a lot is that when the test ends it doesn't clean up after itself it just leaves the app or the emulator in the state like the final state of the test so that's one thing that it's a little annoying so if we go into our test so let's go into our android test here here are the steps that we actually ran so if i click in here this is a neat little option with test project so well yes we have our emulators here and we can watch the test run in the emulators we can also click view device and it will open up a screen mirror to our emulator or to our physical device it just takes it a second to set up mirroring and then once it's done we will see the device this step takes just a second or two ios takes a little bit longer but just before i run this let's just click through our app so if we click menu settings go to home you can see this is the full react native app that i did for our react native presentation i just exported it out of react native into our emulator so now if i want to run this test i will click run and the very first thing it does is it clears the application data and and then down here at the bottom it tells you what it's doing so we click notifications we clicked home and it was very quick now if i go to the left here and i click test i can actually slow this down so we can actually see what's going on so i can go to slow and let's save that and then let's run this again so we reset the app go to notifications go to home go to menu go to settings go back to home one and done and we can even slow that down even more because as we're testing things we really want to see what's going on we want to make sure things are working the way it's supposed to work back to home and test is done now the trick is though once you get done scripting your test you want to run these as fast as possible so that you can actually run multiple tests and get through as much of a regression as possible so now if i kick up the fastest and click run this should be almost lickety split whoops see it went so fast it jumped a step so what's interesting with test project is it will try to find figure out what went wrong and it will try to auto correct itself so you don't have to constantly go back in and say oh okay that ran too fast test failed no what happens is it goes back and says oh i got here but really i was supposed to be here so what happened is it self-repaired so test project kind of has that ai logic in it to determine where things kind of went off the rails and maybe goes back and tries to correct itself and it will tell you when it's doing that so we realized okay maybe running fast was too fast so let's go back to normal and rerun normal the other thing we have here is we have adaptive weights so we can be minimal patient very patient so the adaptive weights if you think back to our selenium talks this is that weight where we can do the implicit or the explicit weight and if we fail just like in test ng and with selenium we can take screenshots of our failures so we can see where things went wrong and if we want we can turn on the automation assistant it's on by default if we turn it off then it won't try to autocorrect if the test fails or if it jumps ahead so let's just run this one more time to make sure i didn't break anything okay good all right so we're going to save and exit out of that test so now let's go look at ios so ios is a little different if we view our device now you have to have the emulator running if you really want to view the device for testing you could come up here and actually view the virtual device however if you do that sometimes xcode and the simulator sometimes they cross talk with each other and they get confused and then one takes ownership over the other and then you can't really run the test if that happens you just have to shut down all the emulators in worst case scenario you would have to come back over here to ios or to your iphone go up to the device and erase all content and settings this basically puts it back to a fresh restart and when you run the test again you'll reinstall the test project agent onto the device and then it will start over but as you can see we're mirroring so if i actually change something here if i go back we will see we go back in the emulator so we can keep this guy minimized and just work within our browser again for ios so we have our name we have the application we're running so if i actually had wanted to test the rest assured project i could deploy it here the problem is because i don't have that apple developer id i could not register it to the device so the device would actually have to be one of the approved devices for testing or simulating a test for a new development app we can set our execution speeds our weights and everything else is the same go back over here to steps and we click run so we start out with our step name we reset the app so we're back to home everything looks good we open up safari yep so figured out i had to go back and reset the browser so we're basically scrolling through the main page of the apple site i'm swiping up i click buy ipad ipad pro and i'm going to swipe up again so see oops can't find it so taking another look so what it's doing and what i think it's doing is it's essentially walking back through the steps to see if one of the previous steps got skipped and then it it determines oh while we're here that's not where we're supposed to be so we went a little too fast so it goes back figures out where it's supposed to be and then it runs the step if the step works then it continues from there if it doesn't and if it's like an id issue it scans the elements on the page here that it can find and see and starts clicking and starts trying to self-repair itself so it looks like this guy's going to fail because either the site has changed or one of the elements has changed so really you don't want to do web page testing on a mobile app to this degree on a site that is totally dependent on maybe angular or randomly changing constantly because of advertisement or you'd be very careful on what you select on the page to click on some other things we can add we can add some test parameters we can add some project parameters we have our locator so we can look up by id xpath tag name just like selenium like we talked about last time and you can explore the path so you can explore the dom tree of the app and walk through and look at the different things and check out the actions the validations the attributes so now that this has changed from what we talked about last time it's just the same except now we're dealing with a mobile app it's pretty cool so i like their the way they treat this they treat it just like a dom just like a web page and you're essentially doing the same type of testing with the mobile app that you would with a web page you
Transcript Segments
[Music]
all right welcome
today we are going to be covering the
second part of test project which is
using test project for mobile testing
we are going to look at how to set up a
mobile device for testing
look at how to create a test project
mobile test we're going to record a
mobile test with a simulator and a
device
and we're going to execute and run the
mobile test
there are some tools you'll need to get
started
if you are going to use test project you
need to install the test project agent
so if you haven't done that already go
back to the previous video on test
project and follow the steps to install
the agent
you will also need android studio
eclipse
some maven dependencies
and xcode and an ios simulator if you're
on a mac
in java
when we export our mobile test or any of
our test project tests to code we're
going to need some java maven
dependencies in order to use these
or run this code from java or from
eclipse we'll need to make sure we
include selenium
selenium java we need to get io java
client and i o test project
for mobile testing
when you export your mobile tests they
export to appium so you'll need the
appium client
we have not discussed appium
in any of our classes to date so i'm
going to keep this very high level when
i discuss appium but i will be doing a
future presentation on how to set up and
run mobile testing with that view in the
future
so let's get started
the slides for now
first
if you're on a mac
you need to do a few things to get
started so the first thing you want to
do is you want to open up xcode
so once xcode loads
you can go under the xcode option and go
to
where is it here it is so xcode open
developer tools you want to open up the
simulator app
now the first time you open this
you want to bookmark it you want to make
sure you
either pin it add it to
create a shortcut to it or something
because really you don't want xcode open
because it's a good memory drain on your
device but you do need the ios simulator
and this can run separate from xcode
the other thing you want to do is you're
going to want to install
if you're doing android development you
want to make sure that you install
android studios
and once you install android studios
and you get it running you need to come
down here to the more action section
if you click that you want to go to the
avd manager
and you need to actually start one of
your virtual devices for android
now if you have a device kicking around
for android
you can go ahead and plug it into your
machine
and test project will recognize your
device as long as it's a fairly recent
device
i found out with the last update that
all my devices are so archaic they won't
load because i can't load the latest
android operating system
so unfortunately running this on device
i won't be able to do today but i will
show you through emulators
the other caveat for running it with
device if you are trying to use ios
and you want to do this on windows you
will need an apple developer license so
you will need to register to get apple
developer id so that you can actually
interact with your device real time
through test project
so that's the setup so if you do those
first and get your emulators running
it's going to make your life a whole lot
easier once you get into test project
so now we open up our browser and we log
into test project
so once we log in it takes us to our
project page where
we have our previous examples for doctor
who and my first test from last
presentation and i have two new tests in
here i have one for ios and one for
android
so i'm going to put this to the right
here
so let's run these so let's run the ios
so when i click run it asks me okay
which project do i want to run this one
i called it my first agent
and i currently have iphone 13 pro
running so i'll slide down to the bottom
here
and for some reason and i don't know why
test project likes to put all your
running emulators or your devices at the
bottom of the list so you have to scroll
all the way to the bottom to find the
one that you want
however once you find the devices you
can
click this and it will show you only the
ones that you want to see
all right so we'll go down we'll take
this guy
click next
one thing that is recommended is that
you always override the default input
parameters unless you have a data source
and for this simple test i'm just going
to override the input parameters and run
with what's in my test
i click run
and while that one's running i'm going
to click run for android because you can
run these in parallel and as you can see
it sees our android device so i will
select this one as well
and click next
and i will again use the defaults
so while these are running i'm going to
switch over to our monitor so we can see
what's going on
so here's our monitor
our tests are preparing
so iphone just reset
so it's reset back to default and so did
android and as you can see on the
android emulator it was actually it's
going to run our default app that we
were running in react native
however apple because i don't have my
apple id it's expired
i couldn't deploy
our sdk or the api to
iphone so instead we're just running a
test testing the safari app going
through and trying to purchase an ipad
pro
so what you're seeing there on the
iphone
is i did a couple slides sliding back
down
and then i'm going to select the 11 inch
display
and then i selected gray and the test
completed so 100 it was now that it's
finished it's going to shut down the
agent
up here is our little monitor this will
tell us what is running so we have no
workers running at the moment so just
finished
and it's done so one of the reasons you
want to make sure that you reset your
device every time you run is when you're
running with emulation one of the things
i've noticed a lot is that when the test
ends it doesn't clean up after itself it
just leaves the app or the emulator
in the state like the final state of the
test
so that's one thing that it's a little
annoying so if we go into our test so
let's go into our android test here
here are the steps that we actually ran
so if i click in here this is a neat
little option with test project so well
yes we have our emulators here and we
can watch the test run in the emulators
we can also click view device and it
will open up a screen mirror to our
emulator or to our physical device
it just takes it a second to set up
mirroring and then once it's done we
will see the device
this step takes just a second or two
ios takes a little bit longer
but just before i run this
let's just click through our app so if
we click menu settings
go to home you can see this is the full
react native app that i did
for our react native presentation
i just exported it out of react native
into our emulator
so now if i want to run this test i will
click run
and the very first thing it does is it
clears the application data and
and then down here at the bottom it
tells you what it's doing so we click
notifications we clicked home
and it was very quick
now if i go to the left here and i click
test i can actually slow this down so we
can actually see what's going on so i
can go to slow
and let's save that
and then let's run this again
so we reset the app go to notifications
go to home
go to menu go to settings
go back to home one and done and we can
even slow that down even more
because as we're testing things we
really want to see what's going on we
want to make sure things are working the
way it's supposed to work
back to home and test is done
now the trick is though once you get
done scripting your test you want to run
these as fast as possible so that you
can actually run multiple tests and get
through as much of a regression as
possible so now if i kick up the fastest
and click run this should be almost
lickety split
whoops see it went so fast it jumped a
step
so what's interesting with test project
is it will try to find figure out what
went wrong and it will try to auto
correct itself
so you don't have to constantly go back
in and say oh okay that ran too fast
test failed
no what happens is it goes back and says
oh
i got here but really i was supposed to
be here
so what happened is it self-repaired so
test project kind of has that ai logic
in it to determine where things kind of
went off the rails and maybe goes back
and tries to correct itself
and it will tell you when it's doing
that
so we realized okay maybe running fast
was too fast so let's go back to normal
and rerun normal
the other thing we have here is we have
adaptive weights so we can be minimal
patient very patient so the adaptive
weights if you think back to
our selenium talks this is that weight
where we can do the implicit or the
explicit weight and if we fail
just like in test ng
and with selenium we can take
screenshots of our failures so we can
see where things went wrong
and if we want we can turn on the
automation assistant it's on by default
if we turn it off then it won't try to
autocorrect if the test fails or if it
jumps ahead
so let's just run this one more time to
make sure i didn't break anything
okay good
all right so we're going to save and
exit out of that test
so now let's go look at ios
so ios is a little different if we view
our device
now you have to have the emulator
running if you really want to view the
device for testing
you could come up here and actually view
the virtual device
however if you do that sometimes xcode
and the simulator sometimes they cross
talk with each other and they get
confused and then one takes ownership
over the other and then you can't really
run the test
if that happens you just have to shut
down all the emulators
in worst case scenario you would have to
come back over here to ios
or to your iphone
go up to the device
and erase all content and settings this
basically puts it back to a fresh
restart
and when you run the test again you'll
reinstall the test project agent onto
the device and then it will start over
but as you can see we're mirroring so if
i actually change something here if i go
back
we will see we go back in the emulator
so we can keep this guy minimized and
just work within our browser
again for ios
so we have our
name
we have the application we're running
so if i actually had wanted to test the
rest assured project
i could deploy it here the problem is
because i don't have that apple
developer id i could not register it to
the device so the device would actually
have to be one of the approved devices
for
testing or simulating a test for a new
development app
we can set our execution speeds our
weights and everything else is the same
go back over here to steps and we click
run
so we start out with our step name
we reset the app so we're back to home
everything looks good we open up safari
yep
so figured out i had to go back and
reset the browser
so we're basically scrolling through the
main page of the apple site i'm swiping
up
i click buy ipad
ipad pro
and i'm going to swipe up again
so see oops can't find it
so taking another look so what it's
doing and what i think it's doing is
it's essentially walking back through
the steps to see if one of the previous
steps got skipped
and then it it determines oh while we're
here
that's not where we're supposed to be so
we went a little too fast so it goes
back figures out where it's supposed to
be and then it runs the step if the step
works then it continues from there
if it doesn't
and if it's like an id issue it scans
the elements on the page here that it
can find
and see and starts clicking and starts
trying to self-repair itself
so it looks like this guy's going to
fail because either the site has changed
or one of the elements has changed
so really you don't want to do
web page testing on a mobile app to this
degree
on a site that is totally dependent on
maybe angular or randomly changing
constantly because of advertisement
or you'd be very
careful on what you select on the page
to click on
some other things we can add we can add
some test parameters we can add some
project parameters
we have our locator so we can look up by
id xpath tag name just like selenium
like we talked about last time
and you can explore the path so you can
explore the dom tree of the app and walk
through and look at the different things
and
check out the actions the validations
the attributes so now that this has
changed from
what we talked about last time it's just
the same except now we're dealing with a
mobile app
it's pretty cool
so i like their the way they treat this
they treat it just like a dom just like
a web page and you're essentially doing
the same type of testing with the mobile
app that you would with a web page
you