Emanuele Meazzo

Using AI self-improvement loops to make raggio Vector Database even faster

Mon, 12 Oct 2026 09:00:00 +0200

The scorecard at the end of my raggio post kept bugging me. raggio won on memory, recall and vector tail latency, but Weaviate took most of the rest, and not by a little: 129 queries per second against 1,050, and a median hybrid search in 104 ms against 36. So that night, a little after midnight and before I’d even hit publish, I opened Claude Code in the raggio repo and typed this:

Analyze the codebase, gather info and reply only no code: does it make sense to port this app to C ? How much effort would that be? Which are the benefits we should expect?

Everyone who writes Python has had this thought at least once: it’s slow, it’s Python, let’s rewrite it in something with a real compiler. Four minutes later the answer came back, and it was no.

raggio: a small ray of RAG

Sun, 27 Sep 2026 09:00:00 +0200

Sooner or later every RAG system sends you the same bill, and it’s written in gigabytes of RAM.

The model wants memory for its weights and its KV cache. The vector database wants memory for every embedding you ever stored, usually as uncompressed floats, usually all loaded at once, whether anyone is asking about them or not. On a big cloud VM you pay and move on. On a machine that also has to run the LLM, or in a department that needs search over a few million documents and owns exactly one modest VM, that bill decides whether the project happens at all.

The Selfish Heart

Thu, 25 Jun 2026 00:01:00 +0100

A few days ago I listened again, after years, to “La ballata dell’amore cieco (o della vanità)” by Fabrizio De André. If you grew up in Italy you know it by heart whether you wanted to or not; for everyone else, the story goes like this.

It happens at least one in the life of a man: he falls madly in love with a woman who does not care for him at all. She doesn’t want his love: she wants to see what it can be made to do. She challenge him: Bring me your mother’s heart, to feed my dogs, he does. Not enough. Then she asks for the blood in his veins, and he gives her that too. And while his life runs out, De André closes the scene on both of them and hands each one an ending. The man dies happy, still in love. The woman watches him die happy and is seized by dismay: nothing is left to her, not his love, not what he owned, only the dried blood of his veins.

Rituals in the modern era: from getting in shape to shaping your life

Sun, 22 Mar 2026 00:01:00 +0100

Bold title uh? Follow me carefully.
One evening, while I was training in the gym, as I’ve been doing for the past 20 years, a girl asked me if she was executing the exercise that her trainer added to her routine correctly: little did she know that experiencing the same scene once again would allow me to finally see and understand one particular pattern I had experienced all my life.

The Problem Solver

Mon, 16 Mar 2026 00:00:00 +0100

When something breaks, most people call someone, I don’t. I open it up. Not because I always know what’s wrong, but because I need to figure it out. And when I look at the problem, the feeling isn’t dread. It’s closer to relief. Finally, something with an answer.

There are plenty of people like this. The “problem solvers”, but that’s too clean a label for what’s actually going on.

Where it comes from

I think there are two roots, and most of us have some mix of both.

A SQL script to automatically infer the data types of data that someone just dumped in your DB with no effort

Wed, 03 Apr 2024 00:00:00 +0200

Do you know the feeling when someone gives you a flat file to import, without any schema information? You have to dump it in your database as-is, then spend precious time figuring out the data type for each column 😥 Let me help you with that, as I’m helping myself too.

Use case:

Someone sends you a flat file (csv), of course they don’t give you ANY information about the data types contained in it
You dump it as-is in your database staging area:
- When using SSMS Flat file importer it can mostly do this job for you, but not 100%
- But if you’re like me and get files from a Data Lake and import it straight away with Azure Data Factory with the “just auto create the table and I’ll deal with it later because I can’t bother right now please” flag on you end up with a table with the right column names but all nvarchar(MAX) columns because the import engine is as lazy as me
At the end of the day you have this table with sub-optimal data types, and here is where this dumb script can help you

Using the power of trial and error and dynamic SQL, I’ve come up with this script which basic idea is simply doing a TRY_CONVERT in various data types (most common defined, you can add more and/or different sizes easily) and checking if the engine was able to convert all of them: it comes up with suggestions about compatible file types for each column, the min/max length for any varchar value and a useful raw data sample too 😉

Quick Tip: Dumb fix for "ORA-26086: direct path does not support triggers" in Azure Data Factory or Azure Synapse Analytics Pipelines

Tue, 09 May 2023 00:00:00 +0200

You should already know that in my opinion, due to my extended experience, Larry Ellison has an explicit agenda to make your life as miserable as possible and locking you in while you use Oracle products.

The latest of the issues I had while moving data around to Oracle using ADF (or really, I’m using Azure Synapse Pipelines, but it’s literally the same thing) is the following error while I was trying to use a copy data activity to move data from a table into another:

How to connect Azure Data Factory / Synapse Pipelines to Google BigQuery without losing your sanity over connection tokens

Wed, 23 Nov 2022 00:00:00 +0100

When I talk about knowing the struggles with the Cloud (as I did recently at PASS 2022), I really mean it, and this is one example of it. This is the case of importing data from Google BigQuery to Azure ADLS v2 (but it can be anywhere else really) using Azure Synapse Pipelines (which is really Azure Data Factory in a different workspace); easy enough, the ugly part is always the authorization part, and in this case having to deal with REST API or JavaScript like those database-hoarding developers.

How to fix "Connect Timeout" and/or "Cannot open server xxx requested by the login" when connecting to Azure SQL Managed Instance in redirect mode

Sat, 14 May 2022 00:00:00 +0200

Here is something that will save you lots of time and headaches when trying to connect to Azure SQL Managed Instances, especially from onprem servers or from other clouds; I had to repeat this multiple times to multiple actors, so I know it will happen to someone else too.

In most cases, “Connect Timeout” and/or “Cannot open server xxx requested by the login; Login failed” errors are caused by the firewall configuration and a lack of understanding the SQLMI networking model, let me explain:

How to fix direct Query authentication delegation issues with Power Bi Report Server with Edge and Chrome

Mon, 12 Jul 2021 00:00:00 +0200

With the end of the IE support for Power Bi (and in general tbh), companies are scrambling finally to move their users from the legacy browser to modern ones; it was about time if you ask me. However, there’s an edge case where using anything but IE is not as straightforward as it could be; in my case Power Bi RS worked fine for any report in any browser, except with direct query reports that were set up to authenticate via Windows Authentication as the user viewing the report:

How to read data from Oracle Database via PowerShell without using ODBC or installing Oracle Client (and import it to SQL Server too)

Tue, 13 Apr 2021 00:00:00 +0200

If you read my previous article on how to configure a Linked server to Oracle , you know that I feel like someone is plotting to keep the topic of how to get our precious data outside of the Oracle ecosystem as obscure as possible out of the oracle circle Fear not! I’m here to get you all the info in order to get data from Oracle Database via Powershell, in a native high-performance way, allowing you to create a multithreaded, reliable and connected ETL flow to feed a data warehouse from Oracle data, like the owner of this blog, or just move quickly some data from one place to another, programmatically, whatever it’s your need.

How to programmatically read from SQL Server extended events files (*.xel) with PowerShell

Wed, 17 Mar 2021 00:00:00 +0100

I love extended events, and if you’re still using SQL profiler, go feel ashamed and come back only when you’ve stopped using it.
Now that you’re familiar with extended events, one of the common destinations for the tracked information are Extended Event Files, where simply put the events are saved to while they happen.

*.XEL files are easy to work with in SSMS, just double click on them and they’ll open up in Management Studio, and you have neat features too, like filtering, aggregations, ecc..
There’s even an export feature that allows you to export the contents to CSV/Table or another XEL file too (it makes sense, because you can export a subset of the original XEL events after filtering)

Slides for "How to use the Cloud for data and actually save money"

Sat, 27 Feb 2021 00:00:00 +0100

On my GitHub you can find the slides for my presentation @ Data Saturdays 001 in Pordenone

How to configure a Linked Server to a modern version of Oracle in SQL Server

Thu, 07 Jan 2021 00:00:00 +0100

It definitely was his plan, all along

Following an Oracle migration to a new version (19c on AWS RDS), I had to update the related linked server in a SQL Instance, upgrading the Oracle Client to the latest version.

Doing so, I’ve come to the conclusion that Oracle purposely makes everything as obscure and overly complicated as possibile, in order to sell support. All the reference articles from the community were outdated and inapplicabile too, of course; lots of guides for 10 years old version, but nothing recent.

SQL Server Diagnostic Notebook Updated

Fri, 18 Dec 2020 00:00:00 +0100

Good news everyone!
As always, I’ve updated the SQL Server Diagnostic Notebook to include the latest updates to the source scripts

No new addition this time around, just updates; but I’m always open to suggestions

The easiest way to access and use it is through the Azure Data Studio Marketplace (or downloading the vsix extension directly from GitHub)

There's a Bug with Extended Events creation in SQL Server Management Studio 18.7: what it is and a workaround

Mon, 26 Oct 2020 00:00:00 +0100

UPDATE 2: The bug was fixed in SSMS 18.8
UPDATE: The bug still isn’t fixed in SSMS 18.7.1

Oh boy, what I found; I haven’t seen anybody else reporting on it, yet, so here I am.
Fortunately, the bug is not breaking, as I’ve found a workaround, but still.

Repro

Open the “New Session…” GUI for extended events in SSMS
Give it a name, it’s not important
Add an event with a Duration Field, I’ve added sp_statement_completed and configure it to be higher than an arbitrary value

Presenting @ DataWeekender the SQL Server Diagnostic Book

Sat, 17 Oct 2020 00:00:00 +0200

I’ve bored you with my SQL Server Diagnostic Book for long enough, now I’ll bore even more people @ DataWeekender , where I’m doing a lightening talk about it, showing people what it is and how to install/use it.

You can download the Slides from the slides Here
My latest post on the Diagnostic book is Here
How to Create a Book Extension for ADS Here

How to ship SQL Jupyter Notebooks and Books directly to Azure Data Studio by creating an extension in the Marketplace

Tue, 22 Sep 2020 00:00:00 +0200

The recently released Extension Generator for Azure Data Studio has opened up yet another way to ship Notebooks and Books to ADS allowing you to easily author your extensions.

In my previous post I’ve detailed how you can now access a Book remotely, now, if you want the book to be actually installed in your ADS instance, you can install it as an extension!

Follow this link for the SQL Server Diagnostic Book as an extension, in this post we’ll look at the feature in a general way.

The SQL Server Diagnostic Book is now on the Azure Data Studio Marketplace!

Tue, 22 Sep 2020 00:00:00 +0200

Great News Everyone!
As detailed in my previous article, you can now create extensions containing SQL (note)books and publish them on the Azure Data Studio Marketplace!
How do I know? Because I did it 🧐

How to install it

As you see in the gif above, the process is super fast:

Be sure to be on Azure Data Studio version 1.22.0 (September 2020) or later
Search for “SQL Server Diagnostic” in the Extensions SearchBox
You’ll find my extension, click on it and/or click “Install” to get the vsix package
- (3rd party extensions can’t be hosted directly in the marketplace directly as of now, so you’ll have to download the extension package from my repository)
Once you’ve downloaded the vsix file from my repo, the last step is to go to File -> “Install Extension from VSIX Package” and select the file you’ve just downloaded
That’s it, profit!

How to use it

Even easier, just open Azure Data Studio command palette (default combination CTRL+SHIFT+P) and start typing “SQL Diagnostic”, you’ll immediately see the command “Launch Book: SQL Server Diagnostic Book” that will immediately load the Jupyter Book in your Notebook tab for you to immediately use 😎

You can now access the SQL Server Diagnostic Book remotely!

Thu, 13 Aug 2020 00:00:00 +0200

UPDATE: This is now somewhat deprecated, because you can install the Book as an extension, without having to type anything, as detailed here.

Good news everyone, thanks to the Azure Data Studio August 2020 update you can now access the SQL Server Diagnostic Book without having to download the book from my Github.

Remote Jupyter Books have been added to ADS, now you just need to click “Add Remote Jupyter Book” in the Notebooks blade and insert my GitHub repository URL, like so:

How to lose hundreds of thousands of dollars by using functions in SQL Server

Tue, 14 Jul 2020 00:00:00 +0200

Ahh, functions, the greatest tool at a programmer’s disposal, they make code reusable and easy to read, they’ve been essential since the first function call was made in the last century…

Unfortunately for the developers out there, when working for SQL Server, especially when tuning for performance, you need to get everything you knew about programming and throw it out of the window, because in SQL we work with sets.

Quick Tip: How to find the address of the PowerBI local tabular instance

Mon, 25 May 2020 00:00:00 +0200

Super quick tip, I’ve seen this done via CMD or DaxStudio, but not via PowerShell (probably because I haven’t searched so much since I’ve done it myself), but here we are:

I’m sure you know that PowerBI tabular models are 100% the same as SSAS tabular models, hence you can use SSMS to connect to a PowerBI Tabular model as if it was SSAS (because it is), to do whatever you want, for example scripting out the objects that you’ve created in the PowerBi Desktop GUI or extract the model to deploy it in SSAS.

You can now script your database objects as a Notebook using SQL Server Management Studio 18.5+

Tue, 07 Apr 2020 00:00:00 +0200

Great News Everyone!

Microsoft just dropped SSMS 18.5 after almost 5 long months without any updates; this new release fixes a lot of bugs and introduces a few new features, above them all I’m now showing you the following.

I’m sure that you used the the “Generate Scripts” feature in SSMS quite a few times, you could generate the code for schema and/or data for any (or all) the objects in your DB, especially if you haven’t embraced that sweet, elusive, devops workflow.

How IT helped the world during the CoronaVirus Pandemic, and how it can continue to help

Mon, 23 Mar 2020 00:00:00 +0100

The 2020 global coronavirus pandemic has shown how Health, Food, and Logistics are essential services to allow the integrity of a nation in a time of need.

However, these weeks of quarantine showed that IT it’s a critical part for the success of the emergency reaction and to be able to continue providing services, here are some examples:

Remote Working

Allowing people to continue to work, and especially still be able to collaborate together, during a time of isolation is certainly one for the most visible results of the internet revolution. Remote working raises both the safety of the employees and the throughput of the companies, which are not required to shut down. Of course, this applies to office work only.

A Self-deployable TICK Stack for ingesting data, monitoring and alerting for any service (including SQL Server)

Wed, 11 Mar 2020 00:00:00 +0100

Oh boy, this is a spinoff of my previous post on “How To Use Grafana (On Docker) To Monitor Your Sql Server (Eventually On Docker Too) – Feat. Influxdb And Telegraf” , which is a nice solution, but I wanted to create something that’s even more easy to deploy, more configurable and without the need of actually knowing influxSQL or learning the influxDB Telegraf schema; for these reasons, I’m going all-in on the TICK Stack:

A PowerBI Report for the COVID19 Emergency in Italy

Sun, 08 Mar 2020 00:00:00 +0100

As you may know, I’m from Italy; at the moment we’re experiencing the most difficult medical emergency in our history due to the new coronavirus.

As the official data from the Italian Government came out, I’ve created a PowerBi report in order to track the current status of the virus.

ITA: Questo report è creato usando i dati ufficiali della protezione civile, si aggiorna ogni sera automaticamente non appena i nuovi dati vengono diffusi.
Il report è interattivo! Vi incoraggio ad esplorare ogni pagina :)

How to use Grafana (on docker) to monitor your SQL Server (eventually on docker too) - feat. InfluxDB and Telegraf

Thu, 13 Feb 2020 00:00:00 +0100

In this container-centric era to complement my SQL Server instance on docker (previous articles here) I’ve looked over containerized monitoring solutions, and chose Grafana as my first candidate; spoiler: everything is simple once you’ve figured out how this stuff works, which can be not trivial for a Windows Guy (like me)

Note: I’m focusing on SQL Server as always but as you will se everything is applicable in the same exact way to any database technology supported by telegraf.

The SQL Diagnostic Notebook: Update 2

Tue, 28 Jan 2020 00:00:00 +0100

Just a quick note, the SQL Diagnostic Book has been updated, here is the changelog:

Updated FirstResponder’s Kit and Glen Berry’s Scripts
Added SQL Assessment API Notebooks
Added sp_whoisactive installation and basic documentation notebooks
Added sp_pressure_detector as a Notebook

That’s it!
As always, to use it just download the folder of the Book from my GitHub (here is a direct link for the folder only, thanks DownGit!) and open the folder with the latest version of Azure Data Studio.

Should you put all your eggs in one basket? Thinking about Database eggs and Cloud baskets

Mon, 20 Jan 2020 00:00:00 +0100

I had a conversation recently that started a train of thoughts related to database, cloud providers, and risk.
Should we treat DBs like investments and diversify investing in the best suit for a specific need, or try to keep everything in the same ecosystem?

Stock market DBs

When your IT stack is diversified, the risk is lowered as you don’t rely on any specific omnipotent provider that can at some moment fail and bring down your entire IT infrastructure, what are the pros and the cons?

How to update a SQL Server container on Docker for Windows (spoiler - there is a catch)

Wed, 08 Jan 2020 00:00:00 +0100

With the 1st CU for SQL 2019 released just yesterday, and Microsoft updating the docker image right away, the only natural response for me was to update the docker instance that I showed you how to deploy a few months back.

The theory

In theory, a docker container can’t be really “updated”, they’re meant to be stateless machines that you spin up and down responding to changes in demand; what we’re technically doing is creating a new container, based on a new image, that has the same configuration and uses the same persistent storage as the old one.

The SQL Diagnostic (jupyter) Book

Wed, 01 Jan 2020 00:00:00 +0100

Welcome to 2020! I wanted to start this year by giving to all my fellow consultants another way to troubleshoot our beloved SQL Servers; I’ve already talked about diagnostic notebooks in the past, and now, since Azure Data Studio has implemented the feature, I wanted to group them into a Diagnostic Book.

As the name implies, a jupyter book is no other than a collection of notebooks (and markdown files) that groups everything in a coherent space, with an index and navigation options alike.

Why RDP'ing into a SQL machine is a bad idea

Thu, 28 Nov 2019 00:00:00 +0100

Guys, you should stop doing this, there are reasons if it’s not recommended to log into the SQL Server Machine via RDP as opposed to using a client on your PC (or another machine).

PS: These recommendations are valid for any DMBS, not only SQL Server

You’re stealing resources from SQL Server

Probably you’re logging into your server because there is a problem, do you really want to steal precious RAM and CPU time by launching SSMS (or any other tool) and making the situation even worse?

Presenting at SQL Saturday

Wed, 20 Nov 2019 00:00:00 +0100

I’ll be presenting my session on “How to use PowerBi as a Free monitoring tool” this Saturday in Parma.

Here you can check the session schedule, there are a lot of interesting sessions; moreover, the event is totally free, if you are in the area and you like data it can’t be missed!

Check out the event page here

Use SQL Server on Docker for Windows, the easy way

Thu, 14 Nov 2019 00:00:00 +0100

I wanted to install and test SQL Server 2019 on a new machine, without the hassle to create a VM from scratch, since I had none of my resources from this new place, that’s when I remembered about Docker and how Microsoft is slowing reaching feature parity for SQL on LInux , moreover, I didn’t need any of the unsupported stuff, so.

I didn’t find a straightforward guide that said “do this and that’s it” and everybody lost themselves in useless words for which I didn’t have the time to read, so, here I am writing it myself.

Watch the Replay of my Session @ GroupBy

Tue, 05 Nov 2019 00:00:00 +0100

https://youtu.be/aP15fp99h2s

Slides and materials for my session "How to use PowerBI as a Monitoring Tool" @ GroupBy.org

Tue, 08 Oct 2019 00:00:00 +0200

Good news everybody!
As you may know if you follow me on Linkedin , I’ve been selected as one of the GroupBy Speakers for the last GroupBy session for 2019!

My session will be: How to use PowerBI as a free monitoring tool (including 5 free reports for SQL Server) (session abstract here)

DeadLock Art

Fri, 04 Oct 2019 00:00:00 +0200

I love when it happens

PROTIP: Performance Tuning on the cloud will save you money by the hour

Mon, 05 Aug 2019 00:00:00 +0200

Note: I'll be focusing on Azure Cloud and SQL Server, but these considerations are valid for any Cloud/DB Vendor.

Whenever people talk about “The Cloud” I often hear: “cloud is expensive” and, sooner or later, “if performance aren’t good we can scale up in minutes later” .
The mindset of taking care of performance issue by scaling up/down cloud resources is a golden egg goose for cloud providers, and a money sink for businesses.

Creating a Report and distributed Alerts for PowerBI Scheduled Refreshes in your organization (starring Powershell, PowerBi & Microsoft Flow)

Mon, 24 Jun 2019 00:00:00 +0200

The Issue

If you ever tried to manage a PowerBI Tenant, you may have come across this problem:
A part the PowerBI success comes from the ability for BI people to self-serve the data it needs, you just provide them with the datasources that they need and they can go around creating all the reports that they want; this is also an issue when trying to manage the tenant, because by default the owner of the dataset is who published the report in the first place, which is usually the BI guy that created it.

How to implement a Linear Regression Prediction in a PowerBI report using Python

Mon, 10 Jun 2019 00:00:00 +0200

Machine learning is the buzzword of the moment, so I wanted to talk about that sweet ML here too.

For a project that I’m working on, I’ve been experimenting on basic ML prediction algorithms, in this instance I’ll show you a basic POC on how ML can be integrated right into PowerBI in a serviceable manner.

Get faster performance and lower network usage in SQL Server Loops by avoiding the "DONE Token" overload

Wed, 22 May 2019 00:00:00 +0200

FYI: You can get the Notebook for this article on my github and experiment yourself (opens with Azure Data Studio).

Preamble

Everybody knows that using loops in SQL Server is not efficient, if you’re able to write that same logic in a set-based statement it’s guaranteed to be faster.
Still, devs can’t be helped, you just can’t seem to nail down the set-equivalent statement for the loop, or, simply, you have to write a loop because the set-statement is too complicated for other members of the team to maintain and you have to bind your query powers for a greater good.

SQL Notebooks: On the right track to enhance Data Analysis on SQL Server

Mon, 29 Apr 2019 00:00:00 +0200

Since about a couple of versions ago, Azure Data Studio introduced Jupyter Notebooks support for SQL Server (and with SQL Server 2019 “Big Data Clusters” on the horizon, it was about time)

What are Notebooks?

“Notebooks” are a tool that our data science/machine learning colleagues know, love and are at their disposal since some time (Python, Spark, etc…);
A notebook is an open-source format that includes both (executable) code and markdown in a single file, in order to have a continuous narrative (be aware that not all features are available in ADS as of now)
Now, Microsoft has implemented the SQL Kernel in ADS.
Moreover, SQL Notebooks aren’t limited to the latest version of SQL Server but you can connect to any supported SQL Server version.

How to identify the unit of measure of Extended Events durations

Wed, 10 Apr 2019 00:00:00 +0200

Even if I use Extended Events almost every day, I always forget the unit of measure of each duration counter, since they’re basically arbitrary; Seconds, milliseconds, microseconds? Whatever, it depends on the dev that implemented that specific counter.

That’s why I’ve added to Tsql.tech Github repository the following code that extracts the descriptions from XE DMVs in order to identify the unit of measure:

SELECT [p].[name] [package_name],
 [o].[name] [event_name],
 [c].[name] [event_field],
 [DurationUnit] = CASE
 WHEN [c].[description] LIKE '%milli%' THEN SUBSTRING([c].[description],
 CHARINDEX('milli', [c].[description]), 12)
 WHEN [c].[description] LIKE '%micro%' THEN SUBSTRING([c].[description],
 CHARINDEX('micro', [c].[description]), 12)
 ELSE [c].[description]
 END,
 [c].type_name [field_type],
 [c].[column_type] [column_type]
FROM sys.dm_xe_objects o
JOIN sys.dm_xe_packages p
ON o.package_guid = p.guid
JOIN sys.dm_xe_object_columns c
ON o.name = c.object_name
WHERE [o].[object_type] = 'event'
 AND [c].[name] = 'duration';

For your convenience, here’s the output of the query above in Azure SQL DB

Quickly move data between instances for testing using only PowerShell

Tue, 02 Apr 2019 00:00:00 +0200

Quick tip, how many times did a dev ask you to move the PROD/QA data to DEV to test a feature? Of course, not the whole DB, which would be too easy via a scripted backup/restore, but only a single table, or subset of table data?

How do you deal with it usually? With the SSMS Import data Wizard? (ugh..) Exporting to flat file and reimporting? Linked Servers? External tools? Restore the whole database in the dev instance and extract the single table? Ignore the dev until it goes away?

Cast any string directly to XML with auto-escaping characters in SQL Server

Mon, 25 Mar 2019 00:00:00 +0100

Recently I had to look up the definition for a bunch of SQL objects and didn’t want to manually retrieve them manually in SSMS (with Create Scripts) or Visual Studio (by searching the object name in my TFS repository).

Since lazyness and automation are the basis of a well done engineering work, I wanted to create a list, where I could basically click on the object that I needed and see the definition right away, without any tool or having to code something externally, of course.

How to Show Better Execution Plans in Azure Data Studio

Fri, 22 Mar 2019 00:00:00 +0100

Azure Data Studio is starting to mature and some neat features are coming to the vscode-based (and now multidatabase and multiplatform) tool.

We got query plans in the app since some time, but to be frank they’re not very pretty and have some issues with dark themes:

Phantom Lines in plans with Dark Theme

Load data directly in SQL Server from a xlsx file, without OLEDB or Excel, using Powershell

Wed, 20 Mar 2019 00:00:00 +0100

I don’t know you, but people from other BU are old fashioned guys that do everything by hand, and then send you an Excel file to “put this data in the database” or “find info related to this codes”.

This is an hassle, especially if you don’t have the Microsoft.ACE.OLEDB provider installed in your machine (or for some reason SSMS doesn’t recognize it or you have the wrong version) and hence you cannot use the Import Data task (which is garbage, anyway).
A quick solution that I used in the past is saving the xlsx file as CSV and then use the flat file import wizard to quickly import the data into my staging DB; Still, some characters may prove problematic in a CSV file and you have to choose the right separator, when you have an Excel file with more than one worksheet you have to save more than one CSV file and don’t get me started in how the Flat File Import Wizard analyzes only the first rows of the file and hence chooses datatypes and nullable columns in a non-reliable way (aka I cannot press Advance->Advance->Advance->Done without looking, unaccepptable)

A SQL Server Permission Report in PowerBI

Mon, 28 Jan 2019 00:00:00 +0100

What it is and how to get it

When working with SQL Instances that have lots of databases, sensitive data and equally lots of users/applications that accesses those, it can get pretty troublesome to keep track of every user, for every database, down to the single column/action.
That’s why I developed this report, in orde to be able to quickly determine the security status of each object or user in the system.

Why identity values can jump by the 1000's ? Is it normal?

Fri, 28 Dec 2018 00:00:00 +0100

It must have happened you at least once to find identity columns with values that from one row to the next one jump by 1000’s rows, of course without having anyone deleting the rows in between, how this occurs?

Identity columns don’t actually recalculate every time the next number to be inserted, SQL Server caches a number of values in order to obtain faster inserts and polls from there. Just imagine, if for every row you would have to calculate the identity value based on the previous one, it would add a (however small) overhead to every insert, that will grow linearly with the number of rows in the insert.

Row Level Security Performance Implications

Sat, 15 Dec 2018 00:00:00 +0100

A conversation had me wondering about performance implications in SQL Server RLS feature, specifically on how to mitigate the performance impact;
TL; DR; ? Since security functions are actually Inline Table Valued Functions (ITVF), if you write them in a decent way the queries will run in the most efficent way possibile, avoiding RBAR processing.

Let’s set the case scene; I’m starting from the ColdRoomTemperatures_Archive table in the WideWorldImporters sample database, why? I already had the DB restored and ready to go, it has a reasonable number of rows and the column ColdRoomSensorNumber it’s a prefect candidate for the example I had in mind.

A quick function to remove (or keep only) string patterns from SQL Server Strings

Sun, 04 Nov 2018 00:00:00 +0100

Working with strings in SQL Server isn’t as easy or powerful as using any programming language, but, I’m sure that more than once you had to to some data manipulation directly in the database, this may help you out, and it doesn’t use Loops, CLR or XML

The idea is to exploit set logic, as opposed to the classic program logic of when you’re usually playing with strings in whenever programming language of choice:

A PowerBI Report for SQL Server Agent Jobs

Sun, 23 Sep 2018 00:00:00 +0200

As my “community service” this month, I’ve worked on this neat (I hope!) PowerBI report (and companion stored procedure) to easily analyze all what you wanted to know about your Agent Jobs execution, but were too afraid to ask to the standard Reports in SSMS.

This is part of the FirstRespondersKit toolkit, which I encourage you to check out.

Installation:

Install the sp_BlitzJobInfo stored procedure wherever you want
- If you install this in the master database you can launch it from any database ( remember, “sp_” stands for “special”, not for “stored procedure”) or you can install it wherever you like.
Open Up the PowerBI template file

Job Performance Analysis PowerBi Report coming soon

Sun, 16 Sep 2018 00:00:00 +0200

It will be part of the FirstResponderKit tool, free for all

How to populate a PowerBI Dataset from a REST API, using PowerBI's REST API, without any code (via Microsoft Flow)

Fri, 10 Aug 2018 00:00:00 +0200

I was wondering if PowerBI could be used for reporting on Live data, not coming from my Database using DirectQuery (or the new super neat composite mode) with with a Streaming Dataset, which I never used before, so, I got to work.

The Proof Of Concept is to take live data from Cryptocurrency value using REST API and pushing them into a PowerBI report.
I’m using the API from CryptoCompare with this super basic call to get the price of BTC in EUR:

SQL Server: How to Find a safe restore point (and who messed up) by reading from the Transaction Log

Fri, 29 Jun 2018 00:00:00 +0200

Panic: Someone has messed up and deleted a super important piece of information in a table! Who did it? Nobody knows. When did it happen? “The data was there this morning” Super useful info, thanks brah.

So you activate your DBA Magic Powers© and try to deal with this stuff yourself: you need to identify which is the exact point where the data has been deleted (in order to minimize data loss) and eventually who did it (to have someone to blame, instruct on query best practices or blackmail, your choice).

A script to automatically align columnstore indexes to enhance segment elimination (and hence performances)

Mon, 04 Jun 2018 00:00:00 +0200

Columstore indexes are a “new” neat data structure that I like, even if technically they’ve been around for years, only recently they’ve become usable by most customers. Let me recap a little bit what we’re talking about, so the point of this will be clearer:

[caption id=“attachment_569” align=“alignnone” width=“634”] Obligatory Microsoft Image Example[/caption]

Troubleshooting SQL Server by analyzing the deviation from average values of Waitstats and Perfmon counters

Wed, 16 May 2018 00:00:00 +0200

Counters and Waitstats are good boys and useful for understanding what’s going on in your SQL instance, but often are pretty useless by themselves if you don’t know what’s a normal value for your server; For example a value of 100 batch requests/Sec is a value to worry about? It depends ¯\_(ツ)_/¯

Tracking Dynamic SQL performance automatically with a wrapper on sp_executesql (Dynamic SQL Inception)

Wed, 02 May 2018 00:00:00 +0200

in some projects I have a lot of Dynamic SQL going on, some queries are fast, some queries are slow, it depends on how the dynamic statement is actually put together ¯\_(ツ)_/¯ Tracking the performance of such statements is usually difficult because you don’t really know “why it’s slow” and you have to interpolate data from the app and the plan cache (if the plan is still there and wasn’t purged due to memory pressure) and try to figure out which was the piece of the offending dynamic SQL and why it did that.

Creating a Perfmon and Filestats reports in PowerBI [Part 2]

Wed, 25 Apr 2018 00:00:00 +0200

In this previous post I detailed the structure of the objects to be created to create a live report in PowerBI to analyze Perfmon and Filestats data (which in my case was already being gathered but not used by the FirstRespondersKit tool); Let’s get to the PowerBI stuff.

Here is the final result that we’ll reach:

I’ll be starting from the existing FirstRespondersKit PowerBI template, but you can start from scratch too, same thing. I’ll not be going through the steps related to the graphs appearance (labels, titles, whatever) as anyone can personalize them as their heart desires. Note: if images seem small on your screen, you can right-click on them and select “Open in another tab” to view them in their majestic original resolution

A clarification on the waitstat: SOS_SCHEDULER_YIELD

Mon, 16 Apr 2018 00:00:00 +0200

Are you one of the people that says “we have a CPU related issue” whenever it sees SOS_SCHEDULER_YIELD popping up? Let me explain why you’re wrong.

In the books online, SOS_SCHEDULER_YIELD is defined simply as:

Occurs when a task voluntarily yields the scheduler for other tasks to execute. During this wait the task is waiting for its quantum to be renewed.

Well, since the SQL Server scheduler is not preemptive, this means:

Revisiting the classics: Minimizing the Impact of DBCC CHECKDB (by Aaron Bertrand) in 2018

Wed, 04 Apr 2018 00:00:00 +0200

SQL Server has a long history, it has been around since the ‘90, more than 20 years have passed, which is a VERY long time as far as the IT world goes; Especially now that Microsoft has stepped up the release schedule for SQL Server (releases are now scheduled yearly) I thought that maybe it’s a good time to go back and revisit the classic articles and recommendations by the Gurus of the SQL Scene to see if those are stil valid.

What's the best way to massively update a big table?

Wed, 21 Mar 2018 00:00:00 +0100

A thing that can happen once in a while in a DW is the need to massively update a column in a table, let’s find out. I’ll be using the same table as last post , the lineitem table of the tpch test by HammerDB, the three contenders are:

Heap
Clustered Index
Clustered Columnstore Index

No other NonClustered index will be included in the base table. You should already know that each NCIndex that contains the updated column in its key (or includes) needs to be updated too, but that’s a no brainer: if I have such index on the table (especially if the updated column is the Key) it’s better to just disable it and rebuild the Index right after the massive update.

SQL Server 2017 CU5 enhances troubleshooting for Intra-Query Parallelism Deadlock

Tue, 20 Mar 2018 00:00:00 +0100

Do you remember this post from not long ago? Apparently, other than still being an issue, with the latest SQL Server 2017 CU there are additional tools to see what’s going on and troubleshoot. Check out the Microsoft post here.

Test: confronting various methods of bulk loading data from a table to another, what's the fastest?

Sun, 11 Mar 2018 00:00:00 +0100

Most of the support request I get involved with can be summarized with the following keywords: “slow” + “datawarehouse” + “ETL” + “Save us” What about thinking about ETL performance before it goes bad? Before the system is in production for some time , the data size reaches a decent size which wasn’t tested in DEV and you find out that the system is not able to scale at all?

Can a single query deadlock itself? Apparently, yes: A curious case of Intra-query Parallelism

Sat, 03 Mar 2018 00:00:00 +0100

Imagine that you’re in a SQL data warehouse in the middle of the night, a single stored procedure is running, is nothing else, and it’s simply doing inserts and updates, one statement at the time, but then.. Deadlock. How can it be? Something else must be running, right? Someone launched something else, or a transaction was left open the day before, or the Russians were spying querying the server in secret, it must be something, right?!?

Scan Performance Showdown: INROW Predicate pushdown VS Batch Mode Adaptive Join vs Columnstore indexes

Mon, 26 Feb 2018 00:00:00 +0100

I was reading Paul White’s post about bitmaps in SQL Server and, other than realizing that I basically know nothing about anything (like a Socrate’s epiphany) I wanted to at least see how this good stuff added up in the latest 7 years. Down with experimenting, we’re doing this LIVE! (Well, it’s live for me, for you it’s still a blog post)

I’ll start by creating two BigInt-only tables and fill it a bunch of numbers, one small and one big, leaving them as heaps for now

Creating a Perfmon and Filestats reports in PowerBI [Part 1]

Sun, 18 Feb 2018 00:00:00 +0100

If you work for a big company, you may have a bunch of instances/databases under your control, you ran whatever monitoring tool has been provided and everybody is happy; But if your’re like me and you’re working for a big number of customers, each one with a different version of SQL and own configuration/security policies, with no monitoring tool directly available, and have to answer the question “what happened to the instance the other night” you may be a little bit overwhelmed.

How to get and decode callstacks in in extended events for recent versions of SQL Server

Sat, 17 Feb 2018 00:00:00 +0100

The other day I was reading about spinlocks and troubleshooting and it mentioned to get the callstack for a certain xevent and to decode it using the windows debugging symbols, however, that material was related to SQL Server 2012 and the steps weren’t basically replicabile for a modern version like SQL 2016/2017 (or even 2014 I think, but haven’tn checked personally), I searched around and didn’t really find something straight forward for just the process as it is now: So, here we are, small guide on how to get and decode callstacks in recent versions of SQL Server.

Create an asynchronous multithreaded workflow on your SQL Server using the Service Broker

Mon, 12 Feb 2018 00:00:00 +0100

As any other *Server, SQL Server is born with concurrency in mind, a lot of users/applications can connect concurrently and do their job, as designed, perfect, but, what if you’re the only user? What if you’re in some kind of data warehouse configuration where you have to ETL, or you are in a maintenance window and have to do a bunch of stuff in that time, or simply you need to launch a task but you don’t want/need it to be blocking for your operations?

THREADPOOL, or, Why is SQL server not responsive even the CPU is basically idle?

Sat, 03 Feb 2018 00:00:00 +0100

It’s a late Friday afternoon, this means that somebody is going to tell you that the server is unresponsive, users are blocked and there is a imperative deadline for the business at 6PM (which you think is probably because they don’t want to be late to the happy hour), so you do what you do best: sigh and log on to see what’s going on with this instance.

You fire up SSMS, and damn, it’s real, you can’t connect! You check the service, it’s up; The network, it’s up; Weirdly enough, you ask your colleague if he’s connected to the instance, yep he’s in

How to avoid the traps when dealing with parallelism

Thu, 01 Feb 2018 00:00:00 +0100

I work with a vendor application which needs to do all sorts of data crunching in some nightly batches, and, depending on the size of data, this process may take a lot of time. Some time a go a colleague said: “we need more parallelism, we want to parallelize all the things! What can you think may go wrong?” This opened a stream of consciousness that could be really handled only by opening an ad-hoc blog about SQL Server performance, issues and maybe simply neat stuff too, so, here we are.

Ready to build your next project with Hugo?

Mon, 01 Jan 0001 00:00:00 +0000

What Users Are Saying About Hugoplate

Mon, 01 Jan 0001 00:00:00 +0000