XenApp cmdlets VS MFCOM – Performance Test
I have been using MFCOM since I began with MetaFrameXP and have always been a huge fan of the SDKs. I loved the fact that Citrix created an API that was pretty simple to learn, for the most part ‘made sense’ and gave you enough power to actually be functional.
When I first heard about the cmdlets at Synergy 2008 I was a bit confused on how useful they would be and if they would offer the power that MFCOM does. I do a lot of coding in C# and I leverage MFCOM there all of the time. Not knowing enough (and still don’t) about Powershell I was not quite sure how well the cmdlets would work.
I had been part of the beta for the cmdlets but wanted to wait until I had the finalized version to run a few benchmarks between it and MFCOM. MFCOM has always had a few performance issues since it is COM based. I won’t go into the details here except to say that whenever performing actions on large groups of data MFCOM could take forever. For example trying to loop through a large number of sessions (8,000+) would take a while if you were pulling back all of the session properties. This is due to MFCOM making a few different calls to get all of the info. While I don’t have the ability to test session enumeration benchmarks in my lab here at home I can benchmark gathering application properties on large groups of apps.
I used 3 different scripts to perform the tests.
2. MFCOM within Powershell to rule out VBScript as a performance factor
3. XenApp cmdlets within Powershell
- I used the XAcmdlet to create applications in increments of 100. I did this and ran the scripts to gather data. I then went to 200,300,400,500.
$i = 1; do {New-XAApplication Notepad$i -ApplicationType ServerInstalled -CommandLine notepad.exe -ServerNames Server1 -Accounts Domain/UserName; $i +=1} while ($i -le 100)
- All of the tests were ran locally on a XenApp 4.5 server that is also the Data Collector.
- This server is a VM running on XenServer 5.0. The hardware is not server class.
- Each script was ran 3 times to get an average.
- I used Get-XAApplicationReport as it actually pulls back all of the app properties. This is comparable to the LoadData function in MFCOM (From what I can tell anyway).
- I had originally been exporting the data to a csv but this added significant amounts of time for each test so I scrapped it and just went with a LoadData for MFCOM (this gets all properties anyway). For the cmdlet test to avoid having the output dumped to the console (which adds time to the script execution) I assign it to an object.
And now for the results:
The results shocked me so much I ran through a bunch of random tests again. Every single time MFCOM via VBScript came out ahead by a few seconds. I am sure on a more powerful machine the numbers would be lower in general but for the most part they stay pretty consistant. I truly expected the cmdlets to blow MFCOM out of the water because from what I can tell they communicate directly with unmanaged IMA calls via the managed .net wrapper. While MFCOM via VBScript seems to come out ahead in this specific performance test the XACmdlets are still far and away easier to use.
I may perform these tests again with Pwershell V2 so I can test WinRM vs COM.
Please let me know if you think I’ve got something completely wrong here. I would also like to see numbers from others running these scripts and would truly like to here from the XA cmdlet developers to see what their internal tests (performance wise) show.
I think to make the comparison a bit more fair, you would have to output all the properties from the mfcom objects.
Also, instead of Get-XAApplicationReport, try using Get-XAApplication, which does not have to resolve the accounts, server names, and pull all the icon data for every app.
But still, this release is a CTP and it has not been optimized. Even thought it connects to the backend directly, the interoperability with unmanaged code provides a lot of opportunities for optimization.
Thanks.
LoadData() does just that though, does it not? I thought a LoadData call in MFCOM pulls back servers and accounts and enumerating through them does not make a separate call to the Server/Account objects?
I agree it isn’t a fair comparison since it is a CTP and I am looking forward to what you guys can do to improve it!
LoadData loads the app specific data (according to the app type), not its associations, which are more expensive to read (accounts, servers, etc).
This is why I suggested to compare it to Get-XAApplication.
I honestly thought that when a LoadData was performed it went ahead and pulled the servers and accounts (along with all of the other data). I will run some more tests to get a better comparison.
Thanks for the info!