Category Archives: Uncategorized

Drill down into Azure Stream Analytics for Real-time insights — Big Data Virtual Chapter Meeting

Online Meeting URL:https://attendee.gotowebinar.com/register/3873663402265937923
RSVPURL: https://attendee.gotowebinar.com/register/3873663402265937923

Azure Stream Analytics is a fully managed low latency stream processing service that takes the complexity out of stream processing by providing a SQL-like language for expressing data transformations and analysis.
Stream Analytics enables you to harness the power of the cloud to easily create, scale, manage your stream processing solution.
It enables you to create live dashboards, notifications etc. on data in motion within minutes, not days. All you have to do is write your stream processing logic in a SQL like language.

In this session learn details on built-in language capabilities such as filtering, temporal semantics with windowing functions, joining multiple streams, using static data with streaming data etc.
Additionally learn how Stream Analytics integrates with other services such as Azure Event Hubs, Azure Machine Learning, Azure SQL DB, Azure Blobs, Power BI etc. to create your end to end real time solution. Walk out with the knowledge to create your first stream processing job.
See some live demos of end to end stream processing solutions to help give you an idea what is possible when you unlock insights from your data in real-time.

Speaker:
Ryan CrawCour is a Senior Program Manager from the Azure Stream Analytics team. He is a data veteran having spent the past 20+ years dealing with all sorts of data at rest in a variety of different database platforms from SQL, to NoSQL, Relational to Document to Graph, he has recently shifted focus to explore the world of new opportunities available through data in motion.

 

Paco Gonzalez — Big Data Virtual Chapter Leader

Introduction to U-SQL Part 3: Querying data where it lives and Advanced Scalable User Extensibility — Big Data Virtual Chapter Meeti

Online Meeting URL:https://attendee.gotowebinar.com/register/7829689746859814146
RSVPURL: https://attendee.gotowebinar.com/register/7829689746859814146
Abstract
Some of U-SQL’s more advanced capabilities are that it can query data where it lives and provides a scalable user extensibility framework. U-SQL not only provides the ability to query data in the Azure Data Lake Storage, but also other data sources including Windows Azure Blob Storage and SQL Server instances in Azure using federated, distributed queries. U-SQL also provides the ability to extend its capabilities with user-defined operators and aggregators that will be executed in the parallel framework so it can scale over any data sizes. This presentation will introduce you to these two advanced capabilities.

Speaker: Michael Rys
Michael has been doing data processing and query languages since the 1980s. Among other things he has been representing Microsoft on the XQuery and SQL design committees and has taken SQL Server beyond relational with XML, Geospatial and Semantic Search. Currently he is working on Big Data query languages such as SCOPE and U-SQL when he is not enjoying time with his family under water, on the ski slopes, or at autocross.

 

Paco Gonzalez — Big Data Virtual Chapter Leader

Introduction to Azure Data Lake — Big Data Virtual Chapter Meeting

Online Meeting URL:https://attendee.gotowebinar.com/register/3040738264471802628
RSVPURL: https://attendee.gotowebinar.com/register/3040738264471802628

Abstract
Azure Data Lake (ADL) makes it easy to get started with massive amounts of data. ADL builds on many years of experience that Microsoft has internally with exabytes of data being processed by thousands of developers, and offers those same powerful tools to the world. With ADL, you can start doing big data in minutes.

Speaker
Saveen Reddy
Data Lake, Lead Program Manager at Microsoft

Azure Automation SQL Server stored procedures

There is no SQL Agent in Azure databases., but… we have azure automation. In the gallery you can browse and find a sample for executing a sql query. I have modified the sample to execute a stored procedure.

I am using it for machine learning… I will post about that later, here goes the code.

<#

.SYNOPSIS

Outputs the number of records in the specified SQL Server database table.

.DESCRIPTION

This runbook demonstrates how to communicate with a SQL Server. Specifically, this runbook

outputs the number of records in the specified SQL Server database table.

In order for
this runbook to work, the SQL Server must be accessible from the runbook worker

running this runbook.Make sure the SQL Server allows incoming connections from Azure services

by selecting ‘Allow Windows Azure Services’ on the SQL Server configuration page in Azure.

This runbook also requires an Automation Credential asset be created before the runbook is

run, which stores the username and password of an account with access to the SQL Server.

That credential should be referenced for the SqlCredential parameter of this runbook.

.PARAMETER SqlServer

String name of the SQL Server to connect to

.PARAMETER SqlServerPort

Integer port to connect to the SQL Server on

.PARAMETER Database

String name of the SQL Server database to connect to

.PARAMETER StoredProcedure

String name of the database table to output the number of records of

.PARAMETER SqlCredential

PSCredential containing a username and password with access to the SQL Server

.EXAMPLE

Use-SqlCommandSample -SqlServer “somesqlserver.cloudapp.net” -SqlServerPort 1433 -Database “SomeDatabaseName” -Table “SomeTableName” – SqlCredential $SomeSqlCred

.NOTES

AUTHOR: System Center Automation Team

LASTEDIT: Jan 31, 2014

#>

workflow SQLForAzure

{

param(

[parameter(Mandatory =$True)]

[string] $SqlServer,

[parameter(Mandatory =$False)]

[int] $SqlServerPort = 1433,

[parameter(Mandatory =$True)]

[string] $Database,

[parameter(Mandatory =$True)]

[string] $StoredProcedure,

[parameter(Mandatory =$True)]

[PSCredential] $SqlCredential

)


#
Get the username and password from the SQL Credential

$SqlUsername = $SqlCredential.UserName

$SqlPass = $SqlCredential.GetNetworkCredential().Password

inlinescript

{


#
Define the connection to the SQL Database

$Conn = New-Object System.Data.SqlClient.SqlConnection(“Server=tcp:$using:SqlServer,$using:SqlServerPort;Database=$using:Database;User ID=$using:SqlUsername;Password=$using:SqlPass;Trusted_Connection=False;Encrypt=True;Connection Timeout=30;”)


#
Open the SQL connection

$Conn.Open()


#
Define the SQL command to run. In this case we are getting the number of rows in the table

$Cmd=newobject system.Data.SqlClient.SqlCommand(“$using:StoredProcedure”, $Conn)

$Cmd.CommandType=[system.Data.CommandType]‘StoredProcedure’

        $Cmd.CommandTimeout=120


#
Execute the SQL command

$Ds=New-Object system.Data.DataSet

$Da=New-Object system.Data.SqlClient.SqlDataAdapter($Cmd)

[void]$Da.fill($Ds)


#
Output the count

$Ds.Tables.Column1


#
Close the SQL connection

$Conn.Close()

}

}

Introducing U-SQL; Part 2 of 2: Scaling U-SQL and doing SQL in U-SQL— Big Data Virtual Chapter Meeting

Online Meeting URL:https://attendee.gotowebinar.com/register/8582635306950556161
RSVPURL: https://attendee.gotowebinar.com/register/8582635306950556161

Speaker: Michael Rys

Abstract

Making Big Data processing easy requires a great developer support that hides the complexity of managing the scale, allows to easily integrate custom code to handle the complex processing requirements ranging from data cleanup to advanced processing of unstructured data, and provides great tool support for the developer to help in the iterative development process. Thus when we at Microsoft introduced the Azure Data Lake, we decided to also include a new language called U-SQL to make Big data processing easy. It unifies the declarative power of SQL and the extensibility of C# to make writing custom processing of big data easy. It also unifies processing over all data – structured, semistructured and unstructured data – and queries over both local data and remote SQL data sources. This presentation will give you an overview on U-SQL, why we decided to build a new language, what its core philosophical underpinnings are as well as show the language in its natural habitat – the development tooling – showing the language capabilities as well as the tool support from starting your first script to analyzing its performance.

 

Bio

Michael has been doing data processing and query languages since the 1980s. Among other things he has been representing Microsoft on the XQuery and SQL design committees and has taken SQL Server beyond relational with XML, Geospatial and Semantic Search. Currently he is working on Big Data query languages such as SCOPE and U-SQL when he is not enjoying time with his family under water, on the ski slopes, or at autocross.

 

Paco Gonzalez — Big Data Virtual Chapter Leader

Cortana’s API, and Project Oxford

Make sure you add this to your current developments since it is moving really fast.

https://www.how-old.net/ was a success.

These API shows what we will be able to do in the short future:

Face APIs

https://gallery.azureml.net/MachineLearningAPI/b0b2598aa46c4f44a08af8891e415cc7

Computer Vision APIs

https://gallery.azureml.net/MachineLearningAPI/Computer-Vision-APIs-2

Text Analytics

https://gallery.azureml.net/MachineLearningAPI/Text-Analytics-2

And the ones I am building demos for, watch for my next SQL Saturdays Presentations.

Speech APIs

https://gallery.azureml.net/MachineLearningAPI/Speech-APIs-2

Check Project Oxford:

https://www.projectoxford.ai/

and LUIS API.

https://www.projectoxford.ai/luis

We are moving so fast J