Monthly Archives: June 2016

Azure Virual Machine CPU Cores Quota

Did you know that by default your Azure subscription has a limit on the number of CPU cores you can consume for virtual machines? Until recently I didn’t. Maybe like you I’d only ever created one of two small virtual machines (VMs) within my MSDN subscription for testing and always ended up deleting them before coming close to the limit.

Per Azure subscription the default quota for virtual machine CPU cores is 20.

To clarify because the relationship between virtual and physical CPU cores can get a little confusing. This initial limit is directly related to what you see in the Azure portal when sizing your VM during creation.

AzureBasicVMsFor example, using the range of basic virtual machines to the right you would hit your default limit with;

  • 10x A2 Basic’s
  • 5x A3 Basic’s
  • 2.5x A4 Basic’s (if it were possible to have 0.5 of a VM)

Fortunately this initial quota is only a soft limit and easily lifted for your production Azure subscriptions.

Before we look at how to solve this problem it’s worth learning how to recognise when you’ve hit the VM CPU core limit, as its not always obvious.

Limit Symptoms Lesson 1

When creating a new VM you might find the some of the larger size options are greyed out without any obvious reason why. This will be because consumed cores + new machine size cores will be greater than your current subscription limit. Therefore ‘Not Available’. Example below.
AzureVMSizesGrey

The other important quirk here is that the size blade will only dynamically alter its colourations and availability state if the CPU cores are currently being used in VMs that are running. If you have already reached your limit, but the VMs are stopped and appearing as de-allocated resources, you will still be able to proceed and deploy another VM exceeding your limit. This becomes clearer in lesson 2 below where the failure occurs.

Limit Symptoms Lesson 2

AzureVMCreateNormErrorIf you are unlucky enough to have exceeded your limit with de-allocated resource and you were able to get past the VM size selection without issue your new VM will now be deploying on your Azure dashboard if pinned. All good right?… Wrong! You’ll then hit this deployment failure alert from the lovely notifications bell. Example on the right.

Note; I took this screen shot from a different Azure subscription where I’d already increased my quota to 40x cores.

This could occur in the following scenario.

  • You have 2x A4 Basic virtual machines already created. 1x is running and 1x is stopped.
  • The stopped VM meant you were able to proceed in creating a third A4 Basic VM.
  • During deployment the Azure portal has now done its sums of all resources covering both stopped and running VM’s.
  • 8x cores on running VM + 8x cores on stopped VM + 8x cores on newly deployed VM. Total 24.
  • This has exceeded your limit by 4x cores.
  • Deployment failed.

In short; this is the difference in behaviour between validation of your VM at creation time vs validation of your VM at deployment time. AKA a feature!

Limit Symptoms Lesson 3

AzureVMCreateWithJSONError2If deploying VMs using a JSON template you will probably be abstracted away from the size and cores consumed by the new VM because in default template this is just hardcodes into the relevant JSON attribute.

Upon clicking ‘Create’ on the portal blade you will be presented with an error similar to the example on the right. This is of course a little more obvious compared to lesson 1 and more helpful than lesson 2 in the sense that the deployment hasn’t been started yet. But still this doesn’t really give you much in terms of a solution, unless you are already aware of the default quota.

Apparently my storage account name wasn’t correct when I took this screen shot either. Maybe another blob post required here covered where the Azure portal is case sensitive and where it isn’t! Moving on.

 

 The Solution

As promised, the solution to all the frustration you’ve encountered above.

AzureHelpAndSupportTo increase the number of CPU cores available to your Azure subscription you will need to raise a support ticket with the Azure help desk… Don’t sigh!… I assure this is not as troublesome as you might think.

Within the Help and Support section of your Azure portal there are a series of predefined menus to do exactly this. Help and Support will be on your dashboard by default, or available via the far left hand root menu.

Within the Help and Support blade click to add a New Support Request. Then follow the prompts selecting the Issue Type as ‘Quota’, your affected Subscription and the Quota Type as ‘Cores per Subscription’.

AzureIncreaseCPUQuota

Once submitted a friendly Microsft human will review and approve the request to make sure it’s reasonable…. Requesting 1000 extra CPU cores might get rejected! For me requesting an additional 30 cores took only hours to get approved and made available. Not 2 – 4 business days as the expectation managing auto reply would have you believe.

Of course I can’t promise this will always happen as quickly so my advise would be; know your current limits and allow for time to change them if you need to scale your production systems.

I hope this post saved you the time I lost when creating 17x VMs for a community training session.

Many thanks for reading.


Controlling Your Windows 10 IoT Core Device

If like me you have a several Raspberry Pi’s doing different things and running different operating systems controlling them can sometimes be a bit of a challenge, especially with a wealth of remote control protocols and command line tools to choose from. In this post I’m specifically exploring the different ways to control your Raspberry Pi running Windows 10 IoT Core.

The thing that surprised me about Microsoft’s “core” flavour of operating systems is just how much they appear on the network like a normal workstation, this might sound silly and very obvious but for the longest time when installing servers in Hyper-V I would shy away from the minimalist core install because I very wrongly assumed this meant you only got a black CMD prompt and nothing more. With Windows 10 IoT Core I can assure you that this is certainly not the case. Apart from the lack of a pretty title Start Menu the operating system (OS) is feature rich and very easy to work with.

Lets look at what we can use to control our Pi and Core OS.

The Old Fashioned Physical Method

PiShot1Just for completeness its probably best to start with the traditional approach of using physical things like a HDMI cable plugged into the back of your monitor plus a keyboard and mouse connected via USB ports. This is obviously very basic and probably not why you’ve been drawn to this blog post. However even via this method you do get a graphical interface on screen with the ability to alter things like the WiFi settings and actually shutdown the device correctly without pulling the plug.

Windows Core Browser Console

Next and by far my favourite feature in Windows 10 IoT Core is the browser based console that can be connected to using the IP address of the device followed by port 8080. Eg: HTTP://192.168.1.2:8080.

If you navigate to the equivalent address of the device on your network you will be prompted for a set of admin credentials and then taken to a lovely bunch of pages which include a wealth of configuration options. When developing this satisfies almost all of my immediate needs for controlling the device and starting up deployed applications.


IoTSetup


The browser console even includes a Task Manager style page of running processes with some pretty real time performance graphs for CPU, RAM, I/O and Network usage. See slide 5 of 16 in the GIF above.

Lets move on before I turn this blog post into a page of screen shots!

Windows IoT Remote Client

IoTClientNext we have the IoT Remote Client app, which as I type is officially still in preview. However this is another really helpful way to control your device. The desktop application available from the Microsoft Store is basically the IoT Core operating system RDP client, so no need for VNC server licencing between Raspbian and Windows to worry about.

IoTRemoteClientStore link: https://www.microsoft.com/en-gb/store/apps/windows-iot-remote-client/9nblggh5mnxz

To get this running in the browser based console I mentioned above first go to the Remote page and tick the box to Enable Windows IoT Remote Server. AKA Terminal Services. Next install the app on and start it up. If all is well it will detect the IoT device on your network and allow you to connect, or just enter the IP address in the box provided. Post authentication you’ll then have a view of your IoT device matching exactly what the HDMI cable can offer. Plus the ability to interact with the device with a keyboard and mouse through the remote session. When starting apps that include something graphical it’s really useful to see the thing. Another use case might be when performing cloud to device messaging, having the received content on screen is nice to see.

This method of remote control is where I actually pulled the first screen shot above from, rather than taking a picture of the monitor displaying the HDMI output.

IoT Dashboard

Another really useful bit of software in the IoT device toolbox is the IoT Dashboard.

Download link: http://go.microsoft.com/fwlink/?LinkID=708576

This walks you through the setting up a new device SD card plus gives you a My Devices window for easily launching the next 2x features detailed below.
IoTDashboard

PowerShell

What can’t you do with PowerShell in the Microsoft world?

Before the IoT Dashboard connecting to the IoT Core device required starting the PowerShell remote commands service. Then setting the items address etc etc. Like this:

1
2
3
4
5
6
7
$MyDeviceIP = "192.168.1.2" #Example
 
Net start WinRM
 
Set-Item WSMan:\localhost\Client\TrustedHosts -Value $MyDeviceIP
 
Enter-PSSession -ComputerName $MyDeviceIP -Credential "localhost\Administrator"

Now if using the IoT Dashboard above from My Devices you have the option just to right click on the auto detected device and select Connect using PowerShell. This handles all the above prerequisites for connecting and only prompts for credentials. Lovely!

PStoPi

Note; both PowerShell methods of connection to the device will require elevated permissions.

Admin File Share

Lastly and again really for completeness we have the traditional UNC path. From the IoT Dashboard this is made easily in My Devices this time by selecting Open Network Share. This brings up a file explorer window to the C$ admin share for browsing the file system on the SD Card.

PiShot5For anybody that is used to seeing the contents of the C drive on a Windows this folder structure will look very familiar. Program File etc.

Other

Sadly despite a few attempts things like Remote Registry access, remote connection to Services using your local snap-in console and Computer Management are not currently supported. The Service to enable Remote Registry simply isn’t there, along with other remote management services on Windows 10 IoT Core. If this changes I’ll be sure to update this post.

Many thanks for reading.


SQL Battleships

When Battleships Met SQL Server, like Bailys meeting Coffee, but better.

Do you have an equal love of the classic two player board game Battleship and also SQL Server database development? If the answer is yes (which it obviously should be) then this is the blog post for you!

The next question that I’m more than happy to answer for you is; how can I combine these 2x loves?… Well my friend, please continue reading to find out.

SQL BattleShips

So, let’s create ourselves a SQL Server database to play a game of Battleships using tables as our grid and ship layout, then let’s use stored procedures as our weapons in which to fire on your opponent (executing statements against their tables). The other unique advantage of playing SQL BattleShips is that both players don’t need to be in the same room verbally querying each other’s grid references and board. In other words, why use English to query this data from a person when you can use SQL to query a database table.

Getting Setup

If you haven’t already done so, please download the zip file from the Downloads page in the database backups section labelled BattleShips.zip. It contains a SQL Server 2014 database backup file of the database called BattleShips. Please restore the database to a suitable SQL instance (and please check the assumption section below). I say a suitable SQL instance because apparently my previous employer didn’t think we needed it on every production server in the organisation!

A Few Assumptions

  • You or the person creating the new game, which doesn’t have to be one of the players has sysadmin access to the SQL Server instance where the Battleships database has been restored

Why? The new game procedure alters various database objects and setups up SQL logins for each player.

  • The SQL instance has database mail setup and is working.

Why? For ease of automation and sending each player details of what to do next. Including login details.

  • The SQL instance is setup for mixed mode authentication.

Why? Each player will have a SQL authenticated login created on the SQL instance in order to be able to connect to the BattleShips database and play with their respective objects. The database users then have a set of permissions defined which prevents each player for seeing each other’s tables (ship board).

  • You and your opponent are aware of how to edit/query database table data, plus views and execute stored procedures with required parameters in SQL Server Management Studio (SSMS).

Why? Because this is SQL BattleShips and it can’t be played using English queries.

Creating A New Game

To create a new game, execute the following procedure adding an email address for each player as required.

1
2
3
4
5
6
USE [BattleShips]
GO
 
EXEC [dbo].[NewGame]
	@Player1EmailAddress = N'bobby.tables@somewhere.com',
	@Player2EmailAddress = N'chuck.norris@somewhere.com'

Using database mail players will then receive details instructions on what to do and where to connect to using a set of SQL credentials provided. For information the body of this email is sorted in the [dbo].[EmailText].

To isolate players from each other as part of the game SQL authenticated user accounts are created for each player which only grants the necessary permissions to the respective player’s databases objects. This is handled using database schema’s and execute only permissions to certain procedures. Players will not have access to any other database on the SQL instance.

If you are a sysadmin you can of course very easily cheat, but this is not in the spirit of the game and it is expected that each player only uses their SQL account provided in the email sent out.

How To Play

Following the instructions provided by email, once you’ve connected to the BattleShips database in SSMS, edit your ‘board’ table. [playerX].[board]. Add all your available ships and close the table editor once complete.

BattleShipsBoardSetupWhen you are happy with the strategic planning (layout) execute the following stored procedure. This will validate your board layout and provide feedback if a ship is the incorrect length or has been missed. If everything is correct your status will be set to ready.

1
EXEC [playerX].[ImReadyToPlay]

Once both players have completed this operation can battle commence! Query the table players if you are unsure of the other player’s status.

1
SELECT * FROM [dbo].[Players]

They may also be ready to play or still setting up their board. If you’re playing in the same room feel free to speak to each other, it’s not against the rules, just not in the sprite of SQL BattleShips.

When both players are ready start taking shots at each other’s ship grids in turn, with player 1 going first. This can be done by executing the following procedure. You’ll of course want to update the gird reference parameter value once you’ve eliminated A01.

1
2
EXEC [player1].[TakeShot]
	@GridReference = 'A01'

Following execution feedback will be returned about the success of your shot and the states of both your board and a grid of which coordinates you have already tried. If you use the same grid reference twice you’ll be warned so your shot isn’t wasted.
BattleShipsShots
After a winner has been established by an opponent sinking all of their rival’s ships (or updating all of their ships with an X for hit). The players table will be BattleShipsPlayersupdated with the winner’s status and the loser will no longer be able to take any more shot.

Happy hunting my friends and please give me your feedback on any epic battles, and maybe the coding!

Many thanks for reading.


Paul’s Frog Blog

Paul is a Microsoft Data Platform MVP with 10+ years’ experience working with the complete on premises SQL Server stack in a variety of roles and industries. Now as the Business Intelligence Consultant at Purple Frog Systems has turned his keyboard to big data solutions in the Microsoft cloud. Specialising in Azure Data Lake Analytics, Azure Data Factory, Azure Stream Analytics, Event Hubs and IoT. Paul is also a STEM Ambassador for the networking education in schools’ programme, PASS chapter leader for the Microsoft Data Platform Group – Birmingham, SQL Bits, SQL Relay, SQL Saturday speaker and helper. Currently the Stack Overflow top user for Azure Data Factory. As well as very active member of the technical community.
Thanks for visiting.
@mrpaulandrew