Simulate user activity with the GHOSTS framework: Client set-up and Timelines
Jun 26, 2020 by Georgi Nikolov | 3252 views
In part I of our look into the GHOSTS framework, we managed to set up the GHOSTS servers on our computer and connect a simple Windows VM, running the client code, to the GHOSTS API server. The next step is to configure properly our Windows Client to simulate the activity of a real user. To do that we will set up multiple programs and tools that can be run automatically and define their behaviour.
Simulate user activity with the ghosts framework : Introduction
The GHOSTS Client Timelines
Any action we want the Client to execute must be defined beforehand in the timeline file included in with the code. This timeline is responsible for determining multiple factors:
- which programs should be launched
- what event is to be triggered with the given program
- starting time and end time of the event
- any extra parameters that have to be passed to the program
- wait periods between different events
The types of actions an automated program can do, vary from simple commands, such as creating a new file, to more complex ones, such as downloading a file from a specified website and unzipping it in a new folder. A simple action is defined by one event, where more complex actions will be executed by chaining multiple events one after the other to accomplish the expected behaviour.
Setting up the Windows Client Timelines:
We will run the GHOSTS client code, downloaded from Carnegie Mellon University repository, on Windows 10 Virtual Machine downloaded from the official Microsoft website. The GHOSTS framework can automate multiple types of program, ranging from internet browsers to command line tools and Microsoft Office. As we don't have a license key for Microsoft Office, for our setup we will focus on creating timelines for Internet Browsers (Chrome, Mozilla Firefox, Internet Explorer) and out-of-the-box tools available on the Windows VM. We will also start by first using the sample timelines that are available at the GHOSTS github repository.
Internet Browser Timelines
Before we jump into configuring the timelines for the different Internet Browsers, we need to take a look at exactly how does the GHOSTS framework automate their use. To accomplish that, GHOSTS uses the Gecko Driver powered by Selenium. The Gecko Driver is used for automatic testing of web code and acts as a proxy between the GHOSTS code and the browser applications. It is important to note that there are certain limitations to the Gecko Driver as it does not support the most recent versions of Chrome and Firefox. It is imperative to check which version of the driver and browsers are to be used and update or downgrade accordingly.
For our testing we needed to downgrade Chrome to version 81.0.4044.129, which is not an easy task as Google doesn't support older versions based on security concerns. It is still possible to find old binaries from 3rd party websites. Once we have installed the appropriate older version, we need to disable the auto update feature of Chrome by following couple of steps:
- from TaskManager remove all Google Processes
- in C:Program Files (x86)Google rename "Update" folder to "Update.disable"
- without the Update folder Chrome can't run auto-update
We also need to install Microsoft Visual C++ Redistributable 2015 to be able to use the GHOSTS Client code with Mozilla Firefox. Luckily at time of testing the latest version of Firefox, 76.0.1 64-bit, is supported by the Gecko Driver, so no downgrade is needed.
- Chrome
After the downgrade we are ready to test the Sample Timeline, shown below, with Chrome.
{
"TimeLineHandlers": [
{
"HandlerType": "BrowserChrome",
"Initial": "about:blank",
"UtcTimeOn": "00:00:00",
"UtcTimeOff": "24:00:00",
"Loop": "True",
"TimeLineEvents": [
{
"Command": "browse",
"CommandArgs": [
"http://www.google.com"
],
"DelayAfter": 900000,
"DelayBefore": 0
},
{
"Command": "type",
"CommandArgs": [
"q",
"test search string"
],
"DelayAfter": 900000,
"DelayBefore": 0
},
{
"Command": "click",
"CommandArgs": [
"btnK"
],
"DelayAfter": 900000,
"DelayBefore": 0
}
]
}
]
}There are couple of important parameters in the json file that are the same, no matter what application we want to automate:
HandlerTypeshows what kind of action we want to run, in this case run the Chrome BrowserInitialis the initial webpage that will be opened when the browser is first startedUtcTimeOnandUtcTimeOffset up the time period when the browser will run (in this case its indefinitely)Loopmeans that the application will loop through the events contrary to running them only onceTimeLineEventsis an Array of events that we want the browser to run- Each event has:
Commandthat has to be executedCommandArgsare the arguments that need to be passed to the commandDelayAfterandDelayBeofreare a time delay for Before and After the command (how long the script has to wait to execute the command)
What the timeline does is it will browse to the given website, in this case "www.google.com", type a given string into the search bar and then click on the Search button. The GHOSTS Client is able to determine where to type the string and which button to click by parsing the DOM of the website and looking for the specific element label (in this case q for the search bar and btnK for the Search button). There is a delay set-up after each command so they don't interfere with each other.
- Firefox
- Now we can look at the sample timeline for Firefox:
{
"TimeLineHandlers": [
{
"HandlerType": "BrowserFirefox",
"Initial": "about:blank",
"UtcTimeOn": "00:00:00",
"UtcTimeOff": "24:00:00",
"Loop": "True",
"TimeLineEvents": [
{
"Command": "random",
"CommandArgs": [
"http://www.google.com",
"https://www.dropbox.com"
],
"DelayAfter": 900000,
"DelayBefore": 0
},
{
"Command": "random",
"CommandArgs": [
{
"Uri": "http://httpbin.org/post",
"Category": "cat1",
"Method": "POST",
"Headers": {
"1": "a",
"2": "b"
},
"FormValues": {
"1": "a",
"2": "b"
}
},
{
"Uri": "http://httpbin.org/put",
"Category": "cat1",
"Method": "PUT",
"Headers": {
"1": "a",
"2": "b"
},
"Body": "body"
},
{
"Uri": "http://httpbin.org/delete",
"Category": "cat1",
"Method": "DELETE"
}
],
"DelayAfter": 900000,
"DelayBefore": 0
}
]
}
]
}In this timeline we can observe two different command events that we can simulate. The first is a typical browsing activity where we can supply a list of different websites and they will be accessed in random order. The second command event is using a websites API to send requests (in this case POST/PUT/DELETE).
- Internet Explorer
The same type of timeline as for Chrome and Firefox can also be used for Internet Explorer by just changing the HandlerType parameter of the timeline file.
Command Line Timeline
We can also set up a timeline to run commands via the Command Line Prompt and simulate different activities.
{
"Status": "Run",
"TimeLineHandlers": [
{
"HandlerType": "Command",
"Initial": "",
"UtcTimeOn": "00:00:00",
"UtcTimeOff": "24:00:00",
"Loop": true,
"TimeLineEvents": [
{
"Command": "net view",
"CommandArgs": [ "arp -a" ],
"DelayAfter": 10,
"DelayBefore": 0
}
]
},
{
"HandlerType": "Command",
"Initial": "",
"UtcTimeOn": "00:00:00",
"UtcTimeOff": "24:00:00",
"Loop": "True",
"TimeLineEvents": [
{
"Command": "once",
"CommandArgs": [
"powershell -command $randUser = (Get-Random -InputObject (Get-Content C:\step\ghosts\config\emails-domain.json)).Trim().split('@')[0].Substring(1); dsquery user -samid $randUser",
""
],
"DelayAfter": 30,
"DelayBefore": 0
}
]
}
]
}In the sample timeline above we can see two types of commands: a net view command and a powershell command. We can provide the type of Command Line tool we want to be used during the simulation and any arguments that the tool may require.
This could facilitate further developing extra simulation capabilities by using different script language such as Python or Perl and running them via the timeline.
Trackable Events
Further it is possible to assign each event an ID that we can use to track the execution of the event via the Grafana UI supplied with the GHOSTS server.
{
"TimeLineHandlers": [
{
"HandlerType": "BrowserChrome",
"Initial": "about:blank",
"UtcTimeOn": "00:00:00",
"UtcTimeOff": "24:00:00",
"Loop": false,
"TimeLineEvents": [
{
"Command": "browse",
"CommandArgs": [ "https://dl.dafont.com/dl/?f=italian_breakfast" ],
"DelayAfter": 0,
"DelayBefore": 0
},
{
"Command": "download",
"CommandArgs": [ "//a[contains(@class, 'dl')]" ],
"TrackableId": "<guid id from trackables table/>",
"DelayAfter": 0,
"DelayBefore": 0
}
]
},
{
"HandlerType": "Command",
"Initial": "",
"UtcTimeOn": "00:00:00",
"UtcTimeOff": "24:00:00",
"Loop": false,
"TimeLineEvents": [
{
"Command": "cd %homedrive%%homepath%\Downloads",
"CommandArgs": [
"powershell expand-archive -Path italian_breakfast.zip -destinationpath x",
"cd x",
"dir"
],
"TrackableId": "<guid id from trackables table/>",
"DelayAfter": 10,
"DelayBefore": 10000
}
]
}
]
}In the timeline above we see 3 events that are linked together to simulate a real activity a user might have: the GHOSTS client code will browse to a specific website, will search the DOM of the website for a html tag containing 'dl' and downloaded the file included in it. Lastly the downloaded file will be unzipped into a new folder and the folder will be accessed. The download and unzipping of the downloaded file have an extra parameter added to the event, namely TrackableId. We can supply an ID for each of these events and when the event is executed a new entry will be created server-side in the Postgres DB, saving information related to this specific event. We can then adapt the Grafana UI to display a visualisation that can track the events we want to observe.
Conclusion
The GHOSTS Framework offers a lot of flexibility when we want to simulate real world activity in a virtual network. The creation of specific timelines gives us a powerful tool to shape and model different types of activities. For now these timelines are static and not mutable, but later on we will explore the possibility to change the timelines in real-time via the use of the GHOSTS Server API.
This blog post is licensed under
CC BY-SA 4.0
