Application Insights and Logging

Featured

Logging is such a common and old topic with a myriad of existing tools and libraries out there that one might think if there is anything else that can be improved or if there is anything to innovate in this area. Well apparently there is! Microsoft has come up with Application Insights. 

What do people log?

First and foremost everyone is interested in errors and exception. When something goes wrong we do want to know as much as possible about the problem. Usually we log the exception message and stack trace.

Next we want to be able to log free form messages for whatever needs and maybe also apply some verbosity level to each message.

Enter Application Insights

Application Insights is not just a simple logging framework and is not marketed as such either. According to the documentation:

Application Insights is an extensible analytics service that monitors your live application. It helps you detect and diagnose performance issues, and understand what users actually do with your app.

Basically is similar to products like Google Analytics and more. Aside from the monitoring capabilities that it offers I found it to be a very useful tool in terms of basic logging needs described above with some additional advantages as well.

It can be used in any type of application (web, desktop) hosted in the cloud or on premise, greenfield or legacy. SDKs exists for C#, Javascript and other platforms.

All the “telemetry” (as they call it) ends up in nice dashboard in Azure Portal. So yes you will need an Azure subscription. However the good news is that there is a free plan that will be enough for most scenarios.

Installation

The first thing you will notice about AI is how fast you can get it up and running.

  • Create an AI resource (a very pretentious name in my opinion) on Azure that will give you an instrumentation key (again very pretentious)
  • Install the NuGet package for the SDK (Microsoft.ApplicationInsights.Web)
  • Put the key obtained at first step into the generated AplicationInsights.config file

And this is the manual process. It can be done automatically from Visual Studio 2015 Update 2 UI. In the end your web.config should look like this:


<?xml version="1.0" encoding="utf-8"?>
<ApplicationInsights xmlns="http://schemas.microsoft.com/ApplicationInsights/2013/Settings">
...
<InstrumentationKey>123456d93-d1d7-4869-9f1e-bcaa9b9fea4f</InstrumentationKey>
</ApplicationInsights>

From this point on all sorts of telemetry will be send to the azure portal in a non blocking way. This is the first dashboard you will see:

ScreenShot002

Now lets see how to this tool helps us for logging purposes.

Logging Exceptions

In order to log the exceptions you have to do…nothing! It just happens and in great detail. You start from the main dashboard and drill down into more and more detail:

ScreenShot003

ScreenShot004

I dare you to show me a log that you have and you have logged more detail for a single exception.

In case you want to catch the exception yourself and do some handling but also log it into AI you can use something like the code below:


var aiClient= new TelemetryClient();
...
try
{
...
}
catch (Exception ex)
{

aiClient.TrackException(ex);

}

Log Message

For instances where you would use something like this in your code:


_logger.LogInfo(message);

with AI you can use a similar line:


aiClient.TrackTrace(message, SeverityLevel.Warning, properties);

Log messages are seen as Traces in the portal.

What did the user do?

This is a question that we often ask users in order to reproduce and fix a problem. Sometimes they can tell you exactly (good luck with that) but most often they just vaguely remember or they might even lead you on the wrong path if their tech level is very low, or they just don’t have time to talk to you or are otherwise unavaialble.

Application Insights has a very nice feature  that shows you everything that happened 5 minutes before and after your unfortunate event. This increases the odds that you no longer need to ask that question because everything is recorded.

ScreenShot005

Search

With all this huge amount of information would good search capabilities are mandatory. Fortunately they exist. You can search most telemetry details based on a string:ScreenShot007

Legacy Logging

A very common scenarios is when you already have an application and you have many lines of logging which obviously you do not want to loose nor do you want to have multiple logging destinations. AI integrates very nicely with existing apps that use log4net in 2 steps:

  • Install NuGet package Microsoft.ApplicationInsights.Log4NetAppender
  • Add the appender configuration in web.config in the lo4net section

Both of these steps will be done automatically when you install the nuget package but sometimes it fails to update the config file and you have to do it manually.

Here is how your config should look:


<log4net >
<appender >
...
</appender>
<root>
<priority value="ERROR" />
<level value="ALL" />
<appender-ref ref="adoParallelForwarder" />
<appender-ref ref="aiAppender" />
</root>
<appender name="aiAppender" type="Microsoft.ApplicationInsights.Log4NetAppender.ApplicationInsightsAppender, Microsoft.ApplicationInsights.Log4NetAppender">
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%message%newline" />
</layout>
</appender>
</log4net>

The log message will appear as traces. Notice that the message severity level appears as well:

legacy tracing

Limitations

While AI give you a lot of stuff out of the box is does have it’s shortcomings.

It only keeps the raw data for 7 days. After this period a lot of details will be lost. But in order to diagnose a problem this would be enough in most cases.

You can not export this data to a destination DB of your choosing in a continuous way without getting the payed subscription.  You can do one offs but that does not help too much. Being able to export is a necessary if you want to keep the data for longer periods of time.

If you are only interested in very basic logging without the additional monitoring features AI has you may finds it’s interface much more complicated than a simple text file.

Conclusions

All in all I think AI is a great tool for web apps that provide a lot of monitoring features and can let you know when something starts to gone wrong giving you a lot of details to diagnose and fix the problem.

Just in terms of logging, unless you have some special needs, I say Application Insights is a great way to implement logging. You install 1 nuget package and that is it. Plus if you work on multiple applications and services you will appreciate having everything easily accessible in one place in Azure portal.

So…is it me…or classic logging starts to become obsolete?

Proper Component Isolation with DIP

I think that anyone can agree that good code is S.O.L.I.D. code, and Dependency Inversion Principle (DIP) is a very important one.

The principle states that:
A. High-level modules should not depend on low-level modules. Both should depend
on abstractions.
B. Abstractions should not depend upon details. Details should depend upon
abstractions.

DIP is very useful when you want to isolate a more abstract type from it’s dependencies (details) by usage of interfaces which receive a concrete type via Dependency Injection pattern. So instead of having a direct dependency like this…

directdependency

…you put an interface between the two components like this:

withinterface

Noting too special until now but what about the physical location of the interface?

Continue reading

Top-down vs Bottom-up TDD

Switching from “normal” coding techniques to TDD can be difficult but once you were able to make the mental switch you realize that there is more substance to TDD than just the usual hype phrase “red, green, refactor”.

Test Driven Development can be approached from a various set of angles (some of which probably I don’t even know…yet). Martin Fowler divided TDD practitioners in two categories classics and mockists (details here)Coming from a background where unit testing heavily relied on mocks for perfect isolation of the tests I consider myself to be a mockist TDDer.

While doing TDD you can practically start anywhere in the system from any component/class you like (assuming here that a basic design is already created, and you have a rough view of the system), however the way you go about it, top-down, bottom-up or from the middle has its peculiarities.

Continue reading

Multiple Asserts in Unit Tests – Is it bad?

Unit test code should be treated like first class code so it should be maintained and refactored to improve its qualities just like any other production code.

In test code one smell is having multiple asserts in a test method which usually indicates that the same test method may be testing multiple scenarios. Take the following example:

        [TestMethod]
        public void Divide_ShouldWorkCorrectly()
        {
            var calculator = new Calculator();

            Assert.AreEqual(2, calculator.Divide(4, 2));
            Assert.AreEqual(4, calculator.Divide(4, 1));
            MSTestExtensions.ExceptionAssert.Throws<DivideByZeroException>(() => calculator.Divide(4, 0));
        }
        

Continue reading

Unit Testing Made Easy with Custom Model Binders

In ASP.NET MVC 3, Microsoft has made unit testing controller actions quite easy. Generally you follow 3 basic steps:

  • setup your dependencies
  • call the controller action you want to test
  • assert something on the properties(usually the Model property) of the returned ActionResult derived object.

However there still is a pain point when the action under test makes use of so called ASP.NET intrinsic objects (Request, Response, Session, Cache, Server etc.) which are hold in the HttpContext instance.

Continue reading