Blog by Bruno Câmara

Exploring AWS WAF rules with OWASP Juice Shop

Mon, 04 Mar 2024 00:00:00 GMT

TL;DR: Consider seriously using a WAF in front of your public web apps/APIs. Also, consider educating your developers about the OWASP Top 10 security list.

When we expose our web applications or APIs to the internet, we are exposing the features that add value to our customers, but we are also exposing our applications vulnerabilities ready to be exploited. We need to be prepared to cope with malicious actors. We should do our best and use the best tools to prevent a cyber attack. This is even more true when we expose an app that we don’t know in detail. Imagine a company that has just acquired another company and the due diligence during the acquisition didn’t include a legacy web app that is being exposed to its customers.

We don’t know how vulnerable the app is until we do some exploratory work, penetration testing, code reviews, threat modelling analysis, etc. Probably it’s a legacy app that nobody touches for years and that has some security holes which can be exploited by a threat actor.

Web Application Firewall (WAF)

As a first line of defense, we can use a Web Application Firewall (WAF). Basically a WAF protects web apps/APIs by inspecting HTTP requests. It acts at the layer 7 (Application Layer of the OSI Model) and it behaves like a reverse proxy with policies and rules attached to it to protect malicious requests, preventing them to flow to our servers or services. Using a WAF doesn’t replace other initiatives that we should be doing in our apps such as code reviews, penetration tests, threat modelling analysis, etc. And most of all, using a WAF doesn’t mean that we don’t need to adopt security engineering practices while writing or deploying our apps. At the end of the day, a WAF is just one more tool at our disposal that we can use to make the life of a threat actor harder.

There are several WAFs we can use, but if our workloads are running in AWS, the natural choice is to use AWS WAF. This is not mandatory. In a previous company I worked for, we were using Cloudflare WAF in front of our AWS and Azure public workloads.

When we start using a WAF in front of our apps, it’s normal that we break some functionality because we start blocking some requests that WAF considers potential malicious, even if they are legit. We should test our apps and select and tweak the WAF rules that must be applied to our web app or API.

We should never forget that a WAF is just the first line of defense - our app should be written with security in mind and assuming breach (Zero Trust), and we have to protect it against malicious actors.

OWASP Top 10

As developers we should be aware of the OWASP Top 10, which represents the list of the most critical security risks to web applications. I would say that it consists of the top 10 concerns that we, developers, should have on our head when developing and writing our apps. (Broken access control, Injection, Cryptographic failures, etc.).

When developing features, developers are so focused on the feature itself that sometimes they forget to add some security checks. That’s when we typically introduce vulnerabilities without even noticing. If we have a code review process in place, we can minimize these risks by having team mates reviewing our pull request and signing off. If we have CI/CD pipelines that run security checks on each change before deploying into production, and prevents deployment if it detects critical or high risk vulnerabilities, even better.

In this post, I don’t want to go into the details of the OWASP Top 10 list. I just want to show a possible setup for developers to be able to explore AWS WAF to protect against these risks. To start exploring WAF, we need a vulnerable app to serve as a playground.

OWASP Juice Shop

OWASP Juice Shop is a vulnerable web application, designed intentionally as insecure, and it can be used as a training tool. The app exposes vulnerabilities from the OWASP Top 10 list, and also other security flaws. What we want is to be able to explore vulnerabilities and to check how AWS WAF behaves to prevent them. Basically we want to use Juice Shop as “guinea pig” to AWS WAF.

To be able to assess how WAF behaves we need to perform two installations of Juice Shop:

One installation without any protection - for this I will use docker locally.
One installation with WAF - for this I will use AWS AppRunner

Installing Juice Shop Locally (Docker)

To install Juice Shop locally I’m going to use docker with the official image bkimminich/juice-shop

docker pull bkimminich/juice-shop

Using default tag: latest
latest: Pulling from bkimminich/juice-shop
07a64a71e011: Pull complete
fe5ca62666f0: Pull complete
...
Digest: sha256:4a80df64f0430b73955b97a653d31adb1f7782417b900c930805c7084b5dfe41
Status: Downloaded newer image for bkimminich/juice-shop:latest
docker.io/bkimminich/juice-shop:latest

And then running it

docker run -e "NODE_ENV=unsafe" --rm -p 3000:3000 bkimminich/juice-shop

info: All dependencies in ./package.json are satisfied (OK)
info: Detected Node.js version v20.10.0 (OK)
info: Detected OS linux (OK)
info: Detected CPU x64 (OK)
info: Configuration default validated (OK)
info: Entity models 19 of 19 are initialized (OK)
info: Required file server.js is present (OK)
info: Required file index.html is present (OK)
info: Required file styles.css is present (OK)
info: Required file main.js is present (OK)
info: Required file polyfills.js is present (OK)
info: Required file runtime.js is present (OK)
info: Required file vendor.js is present (OK)
info: Port 3000 is available (OK)
info: Domain https://www.alchemy.com/ is reachable (OK)
info: Chatbot training data botDefaultTrainingData.json validated (OK)
info: Server listening on port 3000

I’m adding the environment variable NODE_ENV=unsafe to be sure that all challenges are available, even those considered dangerous. Let’s try and open our browser locally on port 3000.

OWASP Juice Shop provides interactive training tutorials, and presents vulnerabilities as challenges that we need to discover. It’s really cool as a training security tool for developers. The first challenge is to discover the Score Board page where you can track your progress.

Installing Juice Shop in AWS AppRunner

To run OWASP Juice Shop in AWS, I’ve chosen AWS AppRunner service. The OWASP Juice docs suggest to use an EC2 instance, but I prefer to use the AppRunner service which is much simpler (we don’t need to setup a VPC).

I am going to configure the AppRunner service to use the official docker image. We need to push this image to a private registry in AWS ECR. I am going to use the aws cli to create a repository juice-shop in my private registry of my aws account.

aws ecr create-repository --repository-name juice-shop

To be able to push images to this new repository, we need to login docker into our ECR registry.

aws ecr get-login-password | docker login --username AWS --password-stdin <your-account-id>.dkr.ecr.us-east-1.amazonaws.com

Check the list of images

docker images

Then we just need to push our image

docker tag 402fefa2b068 <account-id>.dkr.ecr.us-east-1.amazonaws.com/juice-shop:latest

docker push <account-id>.dkr.ecr.us-east-1.amazonaws.com/juice-shop:latest

In the AWS Console we should see the image published

Now we can use this image in our AppRunner Service. Navigate to the AppRunner service and create a new service, specifying the docker image URI for the service

Then just give the name juice-shop and specify the cpu and memory.

For now we can assume all the defaults. We are not associating yet any WAF to our service. Let’s first be sure that we are able to run the service. Just review the info on the last step of the wizard and create and deploy the service. AppRunner will take several minutes to perform its magic to deploy the service and expose it to the internet.

You can click on the link and you should see the Juice Shop

Remember that we are still not running Juice Shop with any WAF.

Setting AWS WAF in Juice Shop App Runner

Now that we have Juice Shop running in AWS, let’s put it behind a AWS WAF. We first need create a Web access control list (Web ACL). A Web ACL defines the set of rules we want to apply to our resource, in this case our AWS AppRunner service. We can associate a web acl to protect other resources such as Amazon CloudFront, Amazon API Gateway, Application Load Balancer, Amazon Cognito, etc. Let’s create a web acl with the name juice-shop-web-acl

In this context I want to start with the common rules to protect against that the OWASP Top 10 list. AWS WAF supports out of the box a set of rules that we can enable.

Under AWS Managed Rules, enable the rules that you want to explore. For now I am going to try the Core Rule set to protect against many of the OWASP Top 10 list, and also the SQL database rule set to protect against the popular SQL injection attack.

The rules are organized into groups and are maintained by AWS and AWS Marketplace sellers (Fortinet, Cloudbric, F5, etc.). For now, we can go with the defaults on the rest of the wizard.

Now we need to associate this Web ACL to our AppRunner service. We could have done this step directly while we were creating the web acl, but we can also configure this association in the AppRunner service. Let’s go to the AppRunner service and change the configuration.

In the Security panel, activate the WAF and select the Web ACL that we created.

Save changes and wait. It takes some time to update the service with the association.

After updated with success, we are now in conditions to start comparing the behavior of Juice Shop with and without WAF.

Testing WAF with SQL Injection

Let’s start by the well known SQL Injection attack in the login page to access to the authenticated part of the web, even without knowing any login credentials. Starting with the unprotected version of the site that I am running locally, go to the login page, and I just need to put ' OR 1=1-- in the Email text box, and in the Password text box we can put any value we want.

And here is the result

Basically I am authenticated as the administrator. The SQL injection attack was a success.

Now let’s try to do the same in the version running in AWS with WAF protection.

When we click in the Login button, nothing happens. Apparently WAF is blocking the request. Let’s confirm it in the Developer Tools.

As soon as we open the developer tools, we can see some console errors, and we notice that a POST request to the login endpoint is being blocked with a 403 Forbidden. Let’s inspect what we got in the wire.

We can check in AWS Web ACL the Traffic Overview

We can even check some sample requests

Here is the detail of the request that was blocked.

The request was blocked by the rule SQLi_BODY. We can say that AWS WAF has prevented the SQL injection attack.

Let’s explore another attack vector.

Testing WAF with Cross-Site Scripting (XSS)

Now let’s try another common attack with XSS. Running locally, let’s go to the new customer page and register a new customer with the name <iframe src="javascript:alert('xss')">. You should notice that there’s a client side validation for the email.

Instead of using the frontend, let’s hit directly the API endpoint using Postman

The API endpoint is not validating the email field, and we were able to register something in the database as a user that is not a valid email address. In fact we are injecting a script. Let’s see in the Administration screen, where the list of users is shown, how this user is being displayed. Navigate as administrator (by using sql injection) to http://localhost:3000/#/administration

We are getting the alert box, which is exactly the script we have injected. The XSS attack was succeeded, and we could run a malicious script on the administration page.

Now let’s repeat the same API request in the version that is running behind WAF in AWS.

We are being blocked with a 403 Forbidden as expected. Let’s check the WAF sample requests.

This request was blocked by the rule CrossSiteScripting_BODY. We can say that WAF has prevented the XSS attack as expected.

We can continue our exploration of other attack vectors by visiting each Juice Shop challenge and check if AWS WAF helps to protect against them in some way.

Conclusion

A Web Application Firewall (WAF) is definitely a tool that we should consider using in front of our public face apps and services. Not having a WAF, is like riding a mountain bike without a helmet. We should always remember that a WAF is our first line of defense, but it doesn’t prevent all attacks. As developers, we should write our apps with the OWASP Top 10 list in mind to be sure that our app is secure, even without a WAF.

AWS WAF is the natural adoption for workloads running in AWS and it supports out of the box many rules to protect against common attacks. The Web ACLs need to be tweaked for the app that is going to be protected.

As a final message, consider seriously using a WAF in front of your public apps. It’s something that you won’t regret, even if it costs you some money. But I’m sure that it will cost you even more if you are a victim of a cyber attack.

Using AWS Cognito for Service-to-Service Authorization in ASP.NET Core

Mon, 12 Feb 2024 00:00:00 GMT

In a Microservices architecture, we typically observe the number of services increasing as a result of an organic growth, and some services start talking with other services. Ideally the inter-services communication should be asynchronous following a publish-subscribe pattern, but in practice we have some services that need to follow a request-reply pattern.

This is typically the case when we have Backends-for-Frontends (BFF) acting as separate API gateways for each front-end client. These BFFs need to talk to other services, typically issuing queries to fetch data, or commands to perform business actions being handled by the downstream service. Most of the time these interactions are made through REST APIs, and we need to have a way to authorize these requests, even if all these services are running within the internal network perimeter with restrictions from the outside world, following a Zero Trust strategy.

Besides BFFs, another scenario where we need to authorize a service is when we have external services running on-premise, outside of the cloud, and there’s no human interaction in these services, for example having a windows service that collects some data from on-premise installations and send it to a service that lives in the cloud.

OAuth 2.0 - Client Credentials Grant flow 🔒

There are several solutions to implement Service-to-Service authorization, known also as Machine to Machine (M2M) authorization, such as API keys, Mutual TLS, OAuth, etc. In this post I want to explore using OAuth 2.0 with the Client Credentials Grant flow in the context of ASP.NET Core and Amazon Cognito.

The OAuth 2.0 Client Credentials Grant flow explained:

When the Client (MyBFF) needs to call the Resource Server (MyService), if it doesn’t have a valid access token yet, makes a request to the Authorization Server (AWS Cognito) passing tthe client_id and client_secret
Amazon Cognito validates the request and returns the access_token with a given expiration in expires_in
MyBFF calls MyService to get some data, passing the access_token as a Bearer token in the Authorization header
MyService validates the request and returns the data if the Bearer token is valid.

Amazon Cognito Setup ☁️

To setup Amazon Cognito for our scenario we need the following resources:

A User Pool - even though we won’t have real users in this pool, a User Pool is the materialization of an Authorization Server in OAuth lingo
A Cognito domain - exposes the authorization server OAuth endpoints in a domain
A Resource Server - it represents MyService. We will have as many resource servers as the number of services.
A Client - it represents the Backend-for-Frontend MyBFF. We will have as many clients as the number of services that are calling other services.

I’m going to use CloudFormation to create all these resources instead of AWS console because:

I love CloudFormation - I don’t care if I’m locked in to AWS. The truth is that CloudFormation is so simple and so powerful that I discard any other IaC alternative when managing resources inside AWS
Amazon Cognito in console is not very intuitive. In fact I think that’s one of the disadvantages of Amazon Cognito, when comparing with other solutions like Auth0 (considered by many as the best solution in the market in this field). I think Amazon Cognito still needs to be polished to remove some friction when used by developers and cloud architects.

Creating a User Pool

  # Create a user pool
  MyServicesUserPool:
    Type: AWS::Cognito::UserPool
    Properties:
      UserPoolName: "MyServicesUserPool"
      # Disable Self-Service Account Recovery
      AccountRecoverySetting:
        RecoveryMechanisms:
          - Name: admin_only
            Priority: 1
      # Disable Self-Service Signup
      AdminCreateUserConfig:
        AllowAdminCreateUserOnly: true
      # Just in case a user is created by admin
      MfaConfiguration: "ON"
      EnabledMfas:
        - SOFTWARE_TOKEN_MFA

The important thing here to notice is that I have disabled all self-service operations from end-users since we won’t have any users registered in the pool. We can check the result in AWS console.

In the Sign-in experience tab

In the Sign-up experience tab

Setting a Cognito domain

  # Creates a domain with oauth endpoints
  MyServicesUserPoolDomain:
    Type: AWS::Cognito::UserPoolDomain
    Properties:
      Domain: replace-your-domain-here
      UserPoolId: !Ref MyServicesUserPool

We are using a AWS Cognito domain: https://[replace-your-domain-here].auth.us-east-1.amazoncognito.com

Note: We could have used a custom domain instead of a Amazon Cognito domain, which requires adding some DNS records to your DNS zone.

Creating a Resource Server for `MyService`

  # Resource Server for MyService
  MyServiceUserPoolResourceServer:
    Type: AWS::Cognito::UserPoolResourceServer
    Properties:
      Identifier: MyService
      Name: MyService
      # Organize scopes
      Scopes:
        - ScopeName: weather_read
          ScopeDescription: Grants the ability to read weather
      UserPoolId: !Ref MyServicesUserPool

We can register whatever scopes we want to support in our service. In this case we are just registering one scope MyService/weather_read that grants the ability to read weather data from MyService (which will expose an endpoint to retrieve weather data).

We can check the result in the App Integration tab

Create a Client for `MyBFF`

  # Client for MyBFF
  MyBFFUserPoolClient:
    Type: AWS::Cognito::UserPoolClient
    DependsOn: MyServiceUserPoolResourceServer
    Properties:
      ClientName: MyBFF
      GenerateSecret: true
      AccessTokenValidity: 1  # Hours
      RefreshTokenValidity: 30  # Days
      AllowedOAuthFlows:
        - client_credentials
      AllowedOAuthFlowsUserPoolClient: true
      AllowedOAuthScopes:
        - MyService/weather_read
      EnableTokenRevocation: true
      ExplicitAuthFlows:
        - ALLOW_REFRESH_TOKEN_AUTH
      UserPoolId: !Ref MyServicesUserPool

Basically we are registering a client, with a secret, which is only allowed to trigger the client_credentials grant flow with the scopes MyService/weather_read. In other words, we are allowing this client MyBFF to talk to resource server MyService.

We can check the result in AWS console, in the App Integration tab

CloudFormation Template for Client Credentials Grant flow

I’m not setting all the Advanced Security features configuration available that we would use in a regular User Pool supporting other flows that deal with real users (Authorization Code, Implicit, etc,). I’m also not enabling AWS Web Application Firewall (WAF), which is something that we would do in a real scenario.

This is the template I am using for the example of this blog post.

AWSTemplateFormatVersion: "2010-09-09"

Description: AWS Cognito with OAuth Client Credentials Grant flow

Resources:
  # Create a user pool
  MyServicesUserPool:
    Type: AWS::Cognito::UserPool
    Properties:
      UserPoolName: "MyServicesUserPool"
      # Disable Self-Service Account Recovery
      AccountRecoverySetting:
        RecoveryMechanisms:
          - Name: admin_only
            Priority: 1
      # Disable Self-Service Signup
      AdminCreateUserConfig:
        AllowAdminCreateUserOnly: true
      # Just in case a user is created by admin, we require MFA
      MfaConfiguration: "ON"
      EnabledMfas:
        - SOFTWARE_TOKEN_MFA

  # Creates a domain with oauth endpoints
  MyServicesUserPoolDomain:
    Type: AWS::Cognito::UserPoolDomain
    Properties:
      Domain: <your-gognito-domain>
      UserPoolId: !Ref MyServicesUserPool

  # Resource Server for MyService
  MyServiceUserPoolResourceServer:
    Type: AWS::Cognito::UserPoolResourceServer
    Properties:
      Identifier: MyService
      Name: MyService
      Scopes:         # Organize scopes
        - ScopeName: weather_read
          ScopeDescription: Grantes the ability to read weather
      UserPoolId: !Ref MyServicesUserPool

  # Client for MyBFF
  MyBFFUserPoolClient:
    Type: AWS::Cognito::UserPoolClient
    DependsOn: MyServiceUserPoolResourceServer
    Properties:
      AccessTokenValidity: 1
      AllowedOAuthFlows:
        - client_credentials
      AllowedOAuthFlowsUserPoolClient: true
      AllowedOAuthScopes:
        - MyService/weather_read
      ClientName: MyBFF
      EnableTokenRevocation: true
      ExplicitAuthFlows:
        - ALLOW_REFRESH_TOKEN_AUTH
      GenerateSecret: true
      RefreshTokenValidity: 30
      UserPoolId: !Ref MyServicesUserPool

We can submit this template as a CloudFormation stack in AWS console and wait for its deployment completion.

At this point we can test the Token endpoint to get a new access_token https://[replace-your-domain-here].auth.us-east-1.amazoncognito.com/oauth2/token

curl --location 'https://replace-your-domain-here.auth.us-east-1.amazoncognito.com/oauth2/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'grant_type=client_credentials' \
--data-urlencode 'client_id=your_client_id' \
--data-urlencode 'client_secret=your_cliend_secret'

Here is the result in Postman

One thing that we can check is the content of the access_token in jwt.io

Things to notice about the access_token issued by AWS Cognito in a Client Crdentials Grant flow:

The subject claim sub (subject) is the client_id
There’s no audience aud claim
There’s a claim token_use with value access
The scope claim has the list of scopes, which in this case includes the scope MyService/weather_read,

`MyService` Setup (Resource Server)

Now it’s time to build the service MyService using ASP.NET Core. I will use Minimal APIs in ASP.NET Core 8.

Let’s start by creating a webapi project for MyService.

dotnet new webapi -o MyService

Navigate to folder MyService, and open the project with your preferred editor. Let’s run it from the command line.

dotnet run --launch-profile https
Building...
info: Microsoft.Hosting.Lifetime[14]
      Now listening on: https://localhost:7062
info: Microsoft.Hosting.Lifetime[14]
      Now listening on: http://localhost:5235
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Development
info: Microsoft.Hosting.Lifetime[0]
      Content root path: C:\temp\MyService

Now let’s open the swagger ui in the browser, and execute the operation GET /weatherforecast to get weather forecast data as a response.

We could do the same with curl

curl -X 'GET' 'https://localhost:7062/weatherforecast'
[{"date":"2024-02-10","temperatureC":50,"summary":"Balmy","temperatureF":121},{"date":"2024-02-11","temperatureC":48,"summary":"Chilly","temperatureF":118},{"date":"2024-02-12","temperatureC":23,"summary":"Mild","temperatureF":73},{"date":"2024-02-13","temperatureC":50,"summary":"Warm","temperatureF":121},{"date":"2024-02-14","temperatureC":-4,"summary":"Chilly","temperatureF":25}]

Now we want to protect MyService to only allow authorized clients with access tokens issued by the AWS Cognito User pool we have created previously.

Let’ check the initial Program.cs (just removing comments from the initial template)

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

var app = builder.Build();

if (app.Environment.IsDevelopment())
{
    app.UseSwagger();
    app.UseSwaggerUI();
}

app.UseHttpsRedirection();

var summaries = new[]
{
    "Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"
};

app.MapGet("/weatherforecast", () =>
{
    var forecast =  Enumerable.Range(1, 5).Select(index =>
        new WeatherForecast
        (
            DateOnly.FromDateTime(DateTime.Now.AddDays(index)),
            Random.Shared.Next(-20, 55),
            summaries[Random.Shared.Next(summaries.Length)]
        ))
        .ToArray();
    return forecast;
})
.WithName("GetWeatherForecast")
.WithOpenApi();

app.Run();

record WeatherForecast(DateOnly Date, int TemperatureC, string? Summary)
{
    public int TemperatureF => 32 + (int)(TemperatureC / 0.5556);
}

And now let’s make the changes to protect the /weatherforecast endpoint. First, and since we want to support JWT tokens, we need to add the nuget package Microsoft.AspNetCore.Authentication.JwtBearer

dotnet add package Microsoft.AspNetCore.Authentication.JwtBearer

And now let’s check the differences in order to protect our endpoint.

using System.Security.Claims;using Microsoft.AspNetCore.Authorization;
var builder = WebApplication.CreateBuilder(args);

// Register Authentication with JWT Bearer tokensbuilder.Services.AddAuthentication().AddJwtBearer();// Register Authorization Policy based on Scopesbuilder.Services.AddAuthorization(options =>{    options.AddPolicy("WeatherReadPolicy", policyBuilder =>    {        policyBuilder.RequireAuthenticatedUser();        policyBuilder.RequireClaim("token_use", "access");        policyBuilder.RequireAssertion(ctx =>            ctx.User.FindFirstValue("scope")?.Split(' ').Any(s => s.Equals("MyService/weather_read")) ?? false        );    });});builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

var app = builder.Build();

// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
    app.UseSwagger();
    app.UseSwaggerUI();
}

app.UseHttpsRedirection();

var summaries = new[] {
    "Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"
};

app.MapGet("/weatherforecast", [Authorize(Policy = "WeatherReadPolicy")] () =>{
    var forecast = Enumerable.Range(1, 5).Select(index =>
        new WeatherForecast
        (
            DateOnly.FromDateTime(DateTime.Now.AddDays(index)),
            Random.Shared.Next(-20, 55),
            summaries[Random.Shared.Next(summaries.Length)]
        ))
        .ToArray();
    return forecast;
})
.WithName("GetWeatherForecast")
.WithOpenApi();

app.Run();

record WeatherForecast(DateOnly Date, int TemperatureC, string? Summary)
{
    public int TemperatureF => 32 + (int)(TemperatureC / 0.5556);
}

These are the changes:

Lines 1-2: Just adding some usings
Lines 6: Registering the Authentication middleware. It requires registering the .well-known url configuration for our user pool, which is in the appsettings.json file (see below)
Lines 7-21: Registering the Authorization middleware, with the policy WeatherReadPolicy to check for the scope MyService/weather_read
Lines 38: Applying the [Authorize] attribute with the policy WeatherReadPolicy in order to protect the endpoint.

In the appsettings.json we need to register the metadata address

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "AllowedHosts": "*",
  "Authentication": {    "Schemes": {      "Bearer": {        "MetadataAddress": "https://cognito-idp.us-east-1.amazonaws.com/<your-cognito-user-pool-id>/.well-known/openid-configuration"      }    }  }}

Let’s call again the endpoint without any authorization, and we should see an error.

curl -X 'GET' 'https://localhost:7062/weatherforecast' -i

HTTP/1.1 401 Unauthorized
Content-Length: 0
Date: Mon, 12 Feb 2024 12:21:58 GMT
Server: Kestrel
WWW-Authenticate: Bearer

As expected, the service returns now a 401 Unauthorized.

Let’s call the endpoint with a valid bearer token returned from the Cognito token endpoint

 curl -X 'GET' 'https://localhost:7062/weatherforecast' -i -H 'Authorization: Bearer <your-access-token>'

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Mon, 12 Feb 2024 12:24:16 GMT
Server: Kestrel
Transfer-Encoding: chunked

[{"date":"2024-02-13","temperatureC":41,"summary":"Cool","temperatureF":105},{"date":"2024-02-14","temperatureC":4,"summary":"Freezing","temperatureF":39},{"date":"2024-02-15","temperatureC":50,"summary":"Cool","temperatureF":121},{"date":"2024-02-16","temperatureC":5,"summary":"Sweltering","temperatureF":40},{"date":"2024-02-17","temperatureC":8,"summary":"Hot","temperatureF":46}]

We got a 200 OK. Now that we have MyService, the Resource Server, behaving as expected, let’s setup the client, i.e, MyBFF.

`MyBFF` Setup (Client)

Let’s repeat the ASP.NET Core setup for the Backend-for-frontend MyBFF.

dotnet new webapi -o MyBFF

Go to folder MyBFF, and using your preferred editor, replace the endpoint logic of /weatherforecast to make a REST API call to MyService in order to get weather forecast data. I am using the Refit library to make the REST call, which means that I need to add the nuget package Refit.HttpClientFactory

dotnet add package Refit.HttpClientFactory

And here is the code.

using Microsoft.AspNetCore.Mvc;using Refit;
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

builder.Services.AddRefitClient<IMyServiceClient>()    .ConfigureHttpClient(client => client.BaseAddress = new Uri("https://localhost:7062"));
var app = builder.Build();

// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
    app.UseSwagger();
    app.UseSwaggerUI();
}

app.UseHttpsRedirection();

app.MapGet("/weatherforecast", ([FromServices] IMyServiceClient myServiceClient) =>{    return myServiceClient.GetWeatherForecast();}).WithName("GetWeatherForecast")
.WithOpenApi();

app.Run();

record WeatherForecast(DateOnly Date, int TemperatureC, string? Summary);interface IMyServiceClient{    [Get("/weatherforecast")]    Task<WeatherForecast[]> GetWeatherForecast();}

Let’s check the differences from the original template:

Lines 1-2: Just adding some usings
Lines 32-38: Take advantage of Refit library and define an interface to represent MyService’s REST API client
Lines 9-10: Registering the REST API client for MyService
Lines 23-26: Call MyService to get weather forecast

At this point, if we run MyBFF and call its endpoint /weatherforecast we should see an error since we are doing an unauthorized call to MyService because we don’t have any logic yet to pass a JWT bearer token when doing the call.

Assuming that MyService is running and listening at https://localhost:7062, let’s also run MyBFF. In MyBFF folder

dotnet run --launch-profile https

Building...
info: Microsoft.Hosting.Lifetime[14]
      Now listening on: https://localhost:7248
info: Microsoft.Hosting.Lifetime[14]
      Now listening on: http://localhost:5006
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Development
info: Microsoft.Hosting.Lifetime[0]
      Content root path: C:\temp\MyBFF

And now let’s call MyBFF endpoint https://localhost:7248/weatherforecast

curl -X 'GET' 'https://localhost:7248/weatherforecast' -i

HTTP/1.1 500 Internal Server Error
Content-Type: text/plain; charset=utf-8
Date: Mon, 12 Feb 2024 11:03:38 GMT
Server: Kestrel
Transfer-Encoding: chunked

Refit.ApiException: Response status code does not indicate success: 401 (Unauthorized).
   at Refit.RequestBuilderImplementation.<>c__DisplayClass14_0`2.<<BuildCancellableTaskFuncForMethod>b__0>d.MoveNext() in /_/Refit/RequestBuilderImplementation.cs:line 288
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Http.RequestDelegateFactory.<ExecuteTaskOfTFast>g__ExecuteAwaited|132_0[T](Task`1 task, HttpContext httpContext, JsonTypeInfo`1 jsonTypeInfo)
   at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
   at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)

HEADERS
=======
Accept: */*
Host: localhost:7248
User-Agent: curl/8.4.0

Note ⚠️: detailed exception information is being returned because I’m running in a Development environment. In Production we should not return any detailed information about the exception.

Basically we are getting a 500 Internal Server Error as a result of a failed call to MyService, which is returning a 401 Unauthorized to MyBFF. We can also check the logs of MyBFF

info: Microsoft.Hosting.Lifetime[0]
      Content root path: C:\temp\MyBFF
info: System.Net.Http.HttpClient.Refit.Implementation.Generated+IMyServiceClient, MyBFF, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null.LogicalHandler[100]
      Start processing HTTP request GET https://localhost:7062/weatherforecast
info: System.Net.Http.HttpClient.Refit.Implementation.Generated+IMyServiceClient, MyBFF, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null.ClientHandler[100]
      Sending HTTP request GET https://localhost:7062/weatherforecast
info: System.Net.Http.HttpClient.Refit.Implementation.Generated+IMyServiceClient, MyBFF, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null.ClientHandler[101]      Received HTTP response headers after 73.9603ms - 401info: System.Net.Http.HttpClient.Refit.Implementation.Generated+IMyServiceClient, MyBFF, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null.LogicalHandler[101]      End processing HTTP request after 92.3572ms - 401fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]      An unhandled exception has occurred while executing the request.
      Refit.ApiException: Response status code does not indicate success: 401 (Unauthorized).
         at Refit.RequestBuilderImplementation.<>c__DisplayClass14_0`2.<<BuildCancellableTaskFuncForMethod>b__0>d.MoveNext() in /_/Refit/RequestBuilderImplementation.cs:line 288
      --- End of stack trace from previous location ---
         at Microsoft.AspNetCore.Http.RequestDelegateFactory.<ExecuteTaskOfTFast>g__ExecuteAwaited|132_0[T](Task`1 task, HttpContext httpContext, JsonTypeInfo`1 jsonTypeInfo)
         at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
         at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
         at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)

To fix it, before calling MyService, we need to be sure that we have a valid access_token to pass it as bearer token in the authorization header. Basically we need the following logic in the REST client:

check if we have a valid access_token:
- if no valid access_token exists, we need to call the Authorization Server passing the client_id and client_secret to get a new access_token
Pass the access_token as a bearer token in the Authorization header.

We can use the IdentityModel library to help us implementing this logic in MyBFF, which means that we need to install the nuget package IdentityModel.AspnetCore

dotnet add package IdentityModel.AspnetCore

Now, we just need to register the Access Token Management Service provided by the IdentityModel library and attach it to the Refit http Client.

builder.Services.AddAccessTokenManagement(options =>
{
    options.Client.Clients.Add("MyBFF", new()
    {
        RequestUri = new Uri(builder.Configuration["AUTH_SERVER_TOKEN_ENDPOINT_URL"]!),
        ClientId = builder.Configuration["AUTH_SERVER_MYBFF_CLIENT_ID"],
        ClientSecret = builder.Configuration["AUTH_SERVER_MYBFF_CLIENT_SECRET"]
    });
});

builder.Services.AddRefitClient<IMyServiceClient>()
    .ConfigureHttpClient(client => client.BaseAddress = new Uri("https://localhost:7062"))
    .AddClientAccessTokenHandler("MyBFF");

Just be sure that you add the following configuration keys to the configuration source that you are using.

AUTH_SERVER_TOKEN_ENDPOINT_URL - For a AWS Cognito domain it should be https://[your-chosen-domain].auth.us-east-1.amazoncognito.com
AUTH_SERVER_MYBFF_CLIENT_ID - the client id
AUTH_SERVER_MYBFF_CLIENT_SECRET - the client secret

Note ⚠️: Do not store secrets in source control

Now let’s run again MyBFF, but this time increasing the log level to debug to see what’s happening inside the access token management service.

dotnet run --launch-profile https --Logging:LogLevel:Default=Debug

Building...
dbug: Microsoft.Extensions.Hosting.Internal.Host[1]
      Hosting starting
info: Microsoft.Hosting.Lifetime[14]
      Now listening on: https://localhost:7248
info: Microsoft.Hosting.Lifetime[14]
      Now listening on: http://localhost:5006
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Development
info: Microsoft.Hosting.Lifetime[0]
      Content root path: C:\temp\MyBFF
dbug: Microsoft.Extensions.Hosting.Internal.Host[2]
      Hosting started

Now let’s call MyBFF /wetherforecast endpoint

curl -X 'GET' 'https://localhost:7248/weatherforecast' -i

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Mon, 12 Feb 2024 11:59:31 GMT
Server: Kestrel
Transfer-Encoding: chunked

[{"date":"2024-02-13","temperatureC":13,"summary":"Cool"},{"date":"2024-02-14","temperatureC":-16,"summary":"Cool"},{"date":"2024-02-15","temperatureC":34,"summary":"Hot"},{"date":"2024-02-16","temperatureC":28,"summary":"Cool"},{"date":"2024-02-17","temperatureC":14,"summary":"Hot"}]

We got a 200 OK and weather forecast data was returned, which means that the REST call to MyService has succeeded. Let’s check the logs

dbug: Microsoft.Extensions.Hosting.Internal.Host[2]
      Hosting started
info: System.Net.Http.HttpClient.Refit.Implementation.Generated+IMyServiceClient, MyBFF, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null.LogicalHandler[100]
      Start processing HTTP request GET https://localhost:7062/weatherforecast
dbug: IdentityModel.AspNetCore.AccessTokenManagement.ClientAccessTokenCache[0]      Cache miss for access token for client: MyBFFdbug: IdentityModel.AspNetCore.AccessTokenManagement.TokenEndpointService[0]      Requesting client access token for client: MyBFFdbug: IdentityModel.AspNetCore.AccessTokenManagement.DefaultTokenClientConfigurationService[0]      Returning token client configuration for client: MyBFFinfo: System.Net.Http.HttpClient.IdentityModel.AspNetCore.AccessTokenManagement.TokenEndpointService.LogicalHandler[100]
      Start processing HTTP request POST https://your-cognito-domain.auth.us-east-1.amazoncognito.com/oauth2/token
info: System.Net.Http.HttpClient.IdentityModel.AspNetCore.AccessTokenManagement.TokenEndpointService.ClientHandler[100]
      Sending HTTP request POST https://your-cognito-domain.auth.us-east-1.amazoncognito.com/oauth2/token
info: System.Net.Http.HttpClient.IdentityModel.AspNetCore.AccessTokenManagement.TokenEndpointService.ClientHandler[101]
      Received HTTP response headers after 549.6799ms - 200
info: System.Net.Http.HttpClient.IdentityModel.AspNetCore.AccessTokenManagement.TokenEndpointService.LogicalHandler[101]
      End processing HTTP request after 556.8406ms - 200
dbug: IdentityModel.AspNetCore.AccessTokenManagement.ClientAccessTokenCache[0]      Caching access token for client: MyBFF. Expiration: 02/12/2024 12:08:31 +00:00info: System.Net.Http.HttpClient.Refit.Implementation.Generated+IMyServiceClient, MyBFF, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null.ClientHandler[100]
      Sending HTTP request GET https://localhost:7062/weatherforecast
info: System.Net.Http.HttpClient.Refit.Implementation.Generated+IMyServiceClient, MyBFF, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null.ClientHandler[101]
      Received HTTP response headers after 15.9003ms - 200
info: System.Net.Http.HttpClient.Refit.Implementation.Generated+IMyServiceClient, MyBFF, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null.LogicalHandler[101]
      End processing HTTP request after 601.1684ms - 200

From the logs we can see that:

The token management service got a cache miss when checking for the access token
The token management service has requested a new token
The REST call to MyService has succeeded
The token management service has cached the access token

Now let’s call it a 2nd time, hoping this time we get a cache hit and reuse the access token. Here are the logs we got this time:

info: System.Net.Http.HttpClient.Refit.Implementation.Generated+IMyServiceClient, MyBFF, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null.LogicalHandler[101]
      End processing HTTP request after 658.9196ms - 200
info: System.Net.Http.HttpClient.Refit.Implementation.Generated+IMyServiceClient, MyBFF, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null.LogicalHandler[100]
      Start processing HTTP request GET https://localhost:7062/weatherforecast
dbug: IdentityModel.AspNetCore.AccessTokenManagement.ClientAccessTokenCache[0]      Cache hit for access token for client: MyBFFinfo: System.Net.Http.HttpClient.Refit.Implementation.Generated+IMyServiceClient, MyBFF, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null.ClientHandler[100]
      Sending HTTP request GET https://localhost:7062/weatherforecast
info: System.Net.Http.HttpClient.Refit.Implementation.Generated+IMyServiceClient, MyBFF, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null.ClientHandler[101]
      Received HTTP response headers after 4.1808ms - 200
info: System.Net.Http.HttpClient.Refit.Implementation.Generated+IMyServiceClient, MyBFF, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null.LogicalHandler[101]
      End processing HTTP request after 4.8902ms - 200

Confirmed! We got a cache hit, and the access_token is being reused. When the token expires, we will get again a cache miss, and a new token will be requested.

In this blog post I am using ASP.NET Core as an example, but the principles used here can be used whatever the programming language you are using. I am also not assuming anything about the host where the services are running - it can be On-Premise, in an EC2 instance, in a AWS Lambda + API Gateway, etc. - it works in whatever host.

Conclusion

In this blog post I tried to show how simple and easy it is to authorize Service-to-Service calls by using OAuth2 with Client Credentials grant flow, using AWS Cognito as the Authorization Server, and ASP.NET Core APIs as both Resource Servers or Clients.

AWS Lambda - HTTP APIs and Concurrent Requests

Thu, 15 Sep 2022 00:00:00 GMT

Are you hosting your HTTP API in a AWS Lambda? Are you aware of AWS Lambda concurrency model regarding HTTP concurrent requests? Let’s investigate.

HTTP APIs

When you build a HTTP API using your preferred language and HTTP framework, you expect that the HTTP framework will live in a process which will handle multiple concurrent requests, right?

When it comes to AWS Lambda that doesn’t happen: one instance of a AWS Lambda processes one HTTP request at a time.

You are maybe asking now: what happens then if a second HTTP request comes in while still processing the first request? Well, the second request will be handled by a different AWS Lambda instance. This 2nd AWS Lambda instance can be a new one, if no instances are ready, or an existing one that is idle (not processing requests) and ready to process the new request (because it has handled a recent previous request). This is all done magically and under the hood by the AWS Lambda service.

Is this really a problem? I don’t think so. However, I think it is counterintuitive at first because it’s not what we are used to when handling HTTP requests. I think it’s a smart design indeed, because the computing becomes more predictable since every instance only handles one request at a time.

So, what are really the consequences of this design? Maybe we will have more cold starts than we were expecting, particularly in cases where we have a burst in concurrent requests.

Cold Starts

A cold start happens when a new AWS Lambda instance is required to be created to handle the request coming in, mostly because there are no AWS Lambda instances ready available. When this happens, the AWS Lambda service needs to spin up a new execution environment with the memory, runtime, and configuration specified in the function. Once finished, AWS Lambda runs any initialization code - in HTTP APIs means running the bootstrap of the framework, typically registering services in Dependency Injection (DI) containers, configuring HTTP midleware, etc (the code in Execute initialization code below).

(Source: AWS)

When an instance finishes to handle a HTTP request, the instance is kept warm and ready to handle another HTTP request. In subsequent HTTP requests, if a warm instance is available, only the Execute handler code is invoked. If no HTTP request comes in for a while, the AWS Lambda service will eventually destroy the instance after a while of inactivity.

We can build something really quick to show this behavior. I am going to ilustrate the steps with a ASP.NET Core 6 Minimal API. You need to have installed the AWS Toolkit for Visual Studio 2022.

Create a new AWS Serverless Application (.NET Core - C#) in Visual Studio 2022
Select the Blueprint ASP.NET Core Minimal API

Change the code to add some delay while serving a HTTP request to give us time to launch new requests.

On file Program.cs replace the line

app.MapGet("/", () => "Welcome to running ASP.NET Core Minimal API on AWS Lambda");

by the following code

app.MapGet("/", async () => {
  // Wait 5 seconds
  await Task.Delay(5000);
  return "Welcome to running ASP.NET Core Minimal API on AWS Lambda\n";
});

Publish to AWS Lambda

Give a name to your stack (a stack is a group of resources representing a serverless app).

You can follow the deployment progress and once finished you should see the Stack view tab
Test your endpoint in the browser

The stack that was created includes an API in AWS API Gateway. You can check the URL in the Lambda page that will trigger the function.

Click on the link and you should wait aprox. 5 seconds until you have the answer in your browser.
Launch 10 concurrent requests

At this point we should have a AWS Lambda instance ready to handle more requests. So let’s launch 10 simultaneous requests to see what happens. Open a bash or a windows subsystem for linux (wsl) and run the following command
```
seq 10 | xargs -I{} -P 10 curl <url_api_gateway>
```
The requests are launched concurrently, wait aprox. 5 seconds, and you will see something like this.
Monitoring concurrent executions

Now it’s time to check what happened in terms of number of instances. Probably you were expecting that the instance that served our first request in browser would be used to serve all the concurrent requests. But as I’ve mentioned before, it’s not what happened. We can check the Monitor -> Metrics tab in the function page.

As you can see the AWS Lambda service has created 10 execution environments.
Check number of cold starts

On total we have made 11 requests, one initially in the browser, and then 10 with curl. Let’s see what the logs are saying (AWS CloudWatch logs).

The intial request (browser) was served by a new execution environment, so we had a cold start. A cold start is easy to identify in the logs by searching for a Init Duration in the REPORT log.

Next, when we launched 10 concurrent requests, the first request comming in reuses the execution environment that was created for the initial request completed in the browser (the instance was kept warm). For the remaining 9 requests the AWS Lambda service has created new 9 execution environments, hence 9 cold starts.

When building synchronous APIs like a HTTP API, we want to avoid cold starts, particularly if it’s a customer facing API, because it will include more latency in the whole request-reply flow. But a cold start is almost inevitable if we want a scale to zero approach and avoiding to pay for resources when there is no activity (HTTP requests).

Minimizing Cold Starts

To minimize the impact of cold start we can:

Reduce the number of cold starts
Reduce the time of a cold start

To reduce the number of cold starts we can use Provisioned Concurrency. Provisioned concurrency initializes a requested number of execution environments so that they are prepared to respond immediately to your function’s invocations. Be aware that configuring provisioned concurrency incurs charges to your AWS account.

To minimize the time spent in a cold start there are some things we can do:

keep our package small
minimize library dependencies
adjust memory function (you can use AWS Lambda Power Tuning)
be lazy in initialization

Can your HTTP API live with Cold Starts?

It really depends. First, we need to measure the cost of a cold start in terms of latency. For example, the logs above are showing that a cold start takes aprox. 500ms (for a very simple scenario). A cold start duration varies based on programming language, runtime, etc.

Second, we need to understand how our API is being used by consumers, who are these consumers, and what are the patterns of usage during the day, day of week, etc. Based on that knowledge we can have an idea of what is the impact of a cold start in the flow in which we are using our function.

If we anticipate problems caused by cold starts, we should analyze what is the cost (money) of configuring provisioned concurrency. Furthermore, consider using Provisioned Concurrency with Application Auto Scaling to adjust the number of provisioned instances based on usage. Nevertheless, we need to be aware that we will be charged by using provisioned concurrency.

AWS Serverless alternatives to AWS Lambda

If we are not confortable with the concurrency execution model provided by AWS Lambda, we can always use some alternatives in AWS:

AWS App Runner - a fully managed container application service with auto scaling based on Max Concurrency - we define the max number of requests that will trigger the scale-up process. The AWS App Runner uses the normal execution model where a container instance can serve multiple concurrent requests. There is always at least one instace provisioned.
Fargate with AWS ECS or AWS EKS (Kubernetes) - We are probably already using Kubernetes or any other container orchestration engine, so we can always use it to host our HTTP API. This route provides more flelibility and full control, however is the route that is more complex.

Conclusion

When analyzing a specific scenario, AWS Lambda is typical the option that I consider first in AWS when going serverless . It’s really simple to use, integrates seamlessly with a lot of AWS Services (SQS, S3, SNS, CloudWatch, API Gateway, etc.), and provides a lot of great features specially when it comes to scalability: scale-up or scale-down (scale to zero).

If we want a “scale to zero” approach, then we need to be aware of cold starts, particularly in case of synchronous APIs, where it can cause some impact in terms of latency. If the scenario being analyzed cannot live with cold starts, I would consider using AWS App Runner which has a very interesting pricing model.

Regarding Kubernetes (AWS EKS), I think it makes totally sense when it’s an initiative enterprise wide. The learning curve is not negligenciable (even when we don’t need to install and administer the cluster like in AWS EKS), and typically we need to spend some time configuring the cluster until we have all the knobs needed. For example, if we want to have auto-scale based on events we might need to have KEDA installed (that’s exactly what Microsoft did with Azure Container Apps).

See you next time. I will continue to explore other scenarios like gRPC and WebSockets.

Visual Studio 2022 - Remote Testing

Tue, 21 Dec 2021 00:00:00 GMT

Several months ago I wrote a blog post explaining how could we debug a unit test running in a Linux container using Visual Studio 2019. In that post I mentioned that the experience is far from being great in VS2019. It was possible, but it required some manual steps.

When we want to debug a unit test running in a Linux container, our expectation is to have the same experience as when debugging locally:

Go to Test Explorer
Right Click on the Test and select Debug
The debugger starts and stops in a breakpoint inside the test

That’s exactly what Visual Studio 2022 brought to us with Remote Testing. Let’s see how it works.

Create a xUnit test project.

Choose .NET 6.0

For demonstrating purposes, let’s create the same test used when demonstrating how to debug in Visual Studio 2019. This is a simple test that asserts that the test is running in a Linux container.

  [Fact]
  public void Test_is_running_in_linux_container() {

      Assert.True(RuntimeInformation.IsOSPlatform(OSPlatform.Linux));

  }

Now add a new file testEnvironments.json to the root of the solution

{
  "version": "1", // value must be 1
  "environments": [
    {
      "name": "WSL-Ubuntu",
      "type": "wsl",
      "wslDistribution": "Ubuntu-20.04"
    }
  ]
}

Basically I am adding a new test environment to run my tests in a WSL Ubuntu 20.04 distribution. To check the list of distributions you can run wsl --list in a command prompt. In Test Explorer you now have a dropdown with the list of test environments. Let’s select the test environment just added, WSL-Ubuntu.

Note: if you don’t see the new environment, then close Visual Studio and launch it again. This is an experimental feature and I have experienced some issues while playing with it. We could also add environments connected to docker containers running locally on Docker Desktop, or even remote environments connected by SSH.

Put a breakpoint in the test that you want to debug and select it in Test Explorer to start debugging it.

Debugger stops at the breakpoint

And the test passes.

Change the environment to < Local Windows Environment > and you will see that the test fails.

Thank you Visual Studio 2022 for greatly simplify this experience of remote testing.

Migrating Azure Pipelines from Classic to YAML

Wed, 20 Oct 2021 00:00:00 GMT

🌲 CI/CD Pipelines as Code

If you have started using Azure DevOps several years ago (maybe its name was Visual Studio Team Services by then), then you are probably still using the Classic Build and Release pipelines to fulfill your CI/CD needs. There is nothing wrong with these Classic Pipelines but the trend over the years has been to follow the Everything as Code approach, including CI/CD Pipelines as Code, and Azure Pipelines also supports it by using YAML Pipelines.

There are obvious advantages in having CI/CD Pipelines as Code, in which its definitions live in a source control repo as YAML text files:

we can use the same code flow for CI/CD, for example using pull requests to make a change in the pipeline and require a review from a specific group of people;
it’s easier to check the history of changes
we can use the same tooling we are used for source control
it’s easier to reuse and share pipelines common blocks
it’s easier to collaborate on pipeline definitions

Over the last few days I have been working in a migration to transform Classic Build and Release pipelines into YAML Pipelines.

🤼‍♂️ Unify Build and Release Pipelines

In the Classic approach the Build and the Release are two different pipelines. The Classic Build doesn’t support multi stages and typically finishes by publishing the build output artifacts. Following is an example of a Classic Build pipeline.

The build artifacts are then consumed by a classic Release pipeline where we have the different stages (DEV, QA, PROD, etc.) to where we want to deploy. Following is an example of a Classic Release pipeline.

The team was not using any dependency between classic release stages, and the deployments to all stages (except to DEV) were being triggered manually, as we can see in the image below.

With the YAML approach we can have multi stages which allows us to have in a single pipeline both the Build (as the first stage) and each of the deployment stages (DEV, QA, PROD. etc.). The main difference is that in the YAML approach we can have just a pipeline definition instead of two in classic (Build definition and Release definition).

We can also specify dependencies between stages, specifying that each deployment stage depends on the build stage, as we can see in the image below.

🏗️ Convert Task Groups into Templates

Using the Classic approach, the team has built several Task Groups over the years, which encapsulate sequences of tasks to run in a classic build or release pipeline.

Using the YAML approach these task groups were converted into YAML templates. We have decided also to use an isolated git repository to host all these templates to allow to reuse them in different projects and/or repos.

The motivation to use YAML templates is the same of Task Groups: encapsulate common logic in templates to be able to use it in several pipeline definitions (we want pipeline definitions to be very thin, delegating the heavy work to templates).

💻 Use Environments in YAML

In the classic approach the team was mostly relying on msdeploy to deploy packages to VMs. With YAML Pipelines we can use the concept of Environment, which is a collection of resources that we can target easily with Deployments from a Pipeline.

We just need to add resources to environments (VMs in this scenario) and specify which checks should be performed.

Adding a VM resource to an environment means that this VM needs to have the Azure Pipeline Agent running on that VM, which means that is much easier to perform some tasks on the target server (in the classic approach these tasks were being performed via msdeploy which is not easy to configure to make it work correctly when interacting remotely from the the build server).

💂 Use Approvals instead of Manual Trigger Deployments

As mentioned earlier, in the classic approach, the team was using Manual Trigger Deployments. As I am writing this post, YAML Pipelines don’t support Manual Trigger Deployments, despite the fact that is a feature requested by many for a long time.

The best workaround to simulate a manual trigger deployment is to use Environment Approval checks with a timeout.

However, be aware that the stage will be in a Waiting state while the approval is pending.

The only drawback with this workaround is that the whole pipeline is shown as Queued while not all stages complete (and also that that the time elapsed is still counting), which might not reflect exactly what the team wants.

If the approval times out, the stage is skipped, and the user has the option to rerun the stage manually. You can play with the timeout value to accommodate your team preference of showing the Pipeline as “Queued” or “Completed” (even if some stages were skipped).

📧 Stage Completed Notification

Finally, some team members are used to be notified when a classic deployment is completed.

However, for YAML Pipelines, in the Notifications page, we have only available the notifications that a stage is waiting for an approval or manual validation.

I hope the events of stage state change will be available in the future. Nevertheless it’s possible to have this notification via Service Hooks.

For now, and since the team was using Slack, we have configured Azure Pipelines Slack App to post a slack message when any stage is completed.

That’s all for now.

C# Static fields inline initialization - Be Lazy

Sun, 04 Apr 2021 00:00:00 GMT

✅ The root cause of the failing test

In my previous post I have described the troubleshooting experience of a failed unit test in CI/CD pipeline which was passing on my machine. Today I want to explore what was causing the failure when the test was running in Release mode, and passing when running in Debug mode.

Basically the root cause was a static field inline initialization that was being initialized before some condition to happen.

✂️ Explaining the issue with a simple scenario

Let’s imagine that we have the following implementation in a class with the responsibility to resolve some URLs.

public class UrlResolver : IUrlResolver {
    private readonly ICache cache;

    public UrlResolver(ICache cache) {
        this.cache = cache ?? throw new ArgumentNullException(nameof(cache));
    }

    public string ResolveUrl(string moduleName) {
        var cacheKey = $"{moduleName}_url_cache_key";
        string moduleUrl = cache.Get(cacheKey) as string;
        if (moduleUrl == null) {
            moduleUrl = ResolveUrlInternal(moduleName);
            cache.Set(cacheKey, moduleUrl);
        }

        return moduleUrl;
    }

    private string ResolveUrlInternal(string moduleName) {
        return $"https://somedomain.com/{moduleName}";
    }
}

In the context of an ASP.NET Core application, this class is used inside controllers and it is being injected in its constructors as an IUrlResolver. We are registering these services in the Startup class, and everything works as expected.

services.AddSingleton<ICache, MyCacheImplementation>();
services.AddSingleton<IUrlResolver, UrlResolver>()

The point here is that the UrlResolver has a dependency of a ICache and it requires an implementation of it. Otherwise we are not able to create an instance of a UrlResolver due the following code in its constructor.

public UrlResolver(ICache cache) {
    this.cache = cache ?? throw new ArgumentNullException(nameof(cache));
}

🦹 Now the legacy code

Now let’s imagine that we have some legacy code that has some static public methods that need need to consume an instance of a UrlResolver. Since these methods are static, one option is to use a static field to keep the UrlResolver instance. But to create an instance of it, we need to pass an instance of an ICache. This legacy code is using a Service Locator pattern to resolve instances of specific services, including the cache service. So, we could do something like this

public static class LegacyModuleUrlResolver {

    private static readonly IUrlResolver urlResolver
      = new UrlResolver(ServiceLocator.GetCache());

    public static string ResolveModuleUrl(string moduleName) {
        return urlResolver.ResolveUrl(moduleName);
    }
}

This code is assuming that when we create the UrlResolver we have an instance of ICache being returned by the Service Locator. Again. In the context of an ASP.NET Core, everything works fine becasue the Service Locator is being initialized and built during the Startup. So, what is the chance of passing a null reference of ICache when creating the static field?. Very unlikely.

👎 And now the failing test

Now, let’s imagine the following test.

[Fact]
public void Some_test_that_should_pass() {
    // Arrange
    string moduleName = "my_module_name";

    var services = new ServiceCollection()
        .AddSingleton<ICache>(Substitute.For<ICache>());
    ServiceLocator.Build(services);

    // Act
    var url = LegacyModuleUrlResolver.ResolveModuleUrl(moduleName);

    // Assert
    url.Should().Be("https://somedomain.com/my_module_name");
}

What do you think? Does it pass? Or does it fail? Well, it really depends of the target framework and configuration. For example, in .NET Core 3.1 the test passes in both Debug and Release mode. In .NET 4.7.2, the test passes in Debug mode, but it fails in Release (exactly the behavior I got in CI/CD pipeline issue).

The main question here is: in what moment the static field inline initialization runs? We should only assume that it will run before it’s needed. But we can’t control when the CLR will run the static initialization. And if the initialization fails due a null instance of ICache, the type will get unused for the whole application with a TypeInitializationException and doesn’t recover from this state, which is not good.

If we use a static constructor the results are consistent and the test starts succeeding in all configurations. But again, we can’t control when the CLR will call the static constructor. We just know that it will run before it’s needed.

👍 Let’s have full control and be lazy

When having these scenarios of depending of some services during the initialization, we may want to have full control when the initialization happens, and we want to initialize just before we really need it (being really really lazy). We can use a different approach with a Lazy<T>.

public static class LegacyModuleUrlResolver {

    private static readonly Lazy<IUrlResolver> urlResolver =
        new Lazy<IUrlResolver>(
            () => new UrlResolver(ServiceLocator.GetCache()),
            LazyThreadSafetyMode.PublicationOnly);

    public static string ResolveModuleUrl(string moduleName) {
        return urlResolver.Value.ResolveUrl(moduleName);
    }
}

By using a Lazy<T> we have full control when the initialization will happen. In this case the initialization will occur the first time the static method ResolveModuleUrl is called, when evaluating the Value property of the lazy instance of urlResolver. I am also using the option LazyThreadSafetyMode.PublicationOnly to not cache the exception if for some reason the initialization fails, to prevent the TypeInitializationException described earlier.

This was a simple explanation why the test was failing. In reality the test was much more complex, involving several assemblies provided by several nuget packages. One of the methods in the test was swallowing exceptions, so it was being completely silence, and the test was failing due an assert. This made the diagnostics even harder. Net-net, sometimes it’s good to have full control when things happen and to be lazy in initialization until we really need it.

Chasing a failed test in CI/CD pipeline

Sun, 14 Mar 2021 00:00:00 GMT

😨 You have broken the CI/CD pipeline

A couple of weeks ago I spent some time investigating a test that started to fail in CI/CD pipeline after I have merged a pull request. When a test starts failing in CI/CD, the faces of the people with commits that triggered that CI/CD run go to a public dashboard, visible by the whole R&D team. My face was there 🤦. Not a great feeling. I’ve assigned the failing test to myself. I’ve put the label I'm on it on the test.

💻 It works on my machine

The test was not new, and it was doing some odd stuff in the arrange part of the test in order to setup some behavior in the database related with nested transactions. My pull request was making changes in the database, including a new version of a stored procedure that is being called in that test. I thought it could be something on the stored procedure itself. The curious thing is that I couldn’t reproduce the error in my machine when I was running the test inside Visual Studio, or even using the same runner app that CI/CD pipeline was using.

🕵️‍♂️ Investigation

After spending some time troubleshooting I got the conclusion that the problem was not in the database changes. So what could it be? With the help of my team, I understood that in CI/CD pipeline the tests run against a release build. Not only release, but also a build with an obfuscation process in place. So the next step was to run the test in the exact some conditions. And guess what - the test started to fail in my machine. It should be the obfuscation process, I thought. Let’s remove obfuscation out of the equation - the test was still failing with just a release build without obfuscation.

Next step was to try to understand the differences between the Build and Release configuration. We also checked what were the differences at the .Net Intermediate Language (IL). Nothing relevant from these differences that could explain the behavior of the test.

One member of my team then have tried to apply the NoOptimization attribute to the test method. Then the test started to pass again 💥.

[Fact]
[MethodImpl(MethodImplOptions.NoOptimization)]
public void Test_method_here() {
  // ... test code goes here
}

From the documentation we can see what the NoOptimization means

The method is not optimized by the just-in-time (JIT) compiler or by native code generation (see Ngen.exe) when debugging possible code generation problems.

With this change, my face was not anymore on the public dashboard. But I was not happy with the fix. This code was just a test, but it’s not hard to imagine that some production code could have the same issue. The NoOptimization could be only masquerading the real issue. It seems that the JIT compiler is doing something different for the release build. I plan in my next post to explain what the test was doing, why it was failing in release, and how I have fixed it without the need to rely on compiler services options.

VS 2019 on Windows - Debugging a unit test running in a Linux container

Mon, 11 Jan 2021 00:00:00 GMT

Docker Support in Visual Studio 2019 is great, providing a great experience for running and debugging ASP.NET and Console applications in docker containers. But when it comes to Unit Test projects the experience is not so great.

My expectation was that the experience of running or debugging unit tests in containers would be exactly the same as ASP.NET or Console apps. I was even expecting that I could use the VS Test Explorer: as long as the Docker launch profile was selected, I thought that the tests would be running/debugging inside the container. Unfortunately it’s not the case.

So, imagine that you are working with Visual Studio 2019 on Windows, building a .NET Core project that would be deployed in Windows, but also in Linux, and a specific test is failing in Linux, and you want to debug it. How can we do it in a simple way?

Let’s create a xUnit (my test framework of choice) test project.

For demonstrating purposes, let’s create a simple test that asserts that the test is running in a Linux container.

  [Fact]
  public void Test_is_running_in_linux_container() {

      Assert.True(RuntimeInformation.IsOSPlatform(OSPlatform.Linux);

  }

Unlike ASP.NET projects, in Unit Test projects we don’t have the option “Enable Docker Support” while creating the project. Though, the option is still available via Solution Explorer. Let’s try it.

Choose Linux as target. The result: a Dockerfile is added to the solution and a new Docker launch profile is added to the list of profiles.

This is exactly the same behavior we get in an ASP.NET or Console project. However, the running experience is not the same: the integration between the test explorer and the docker container does not work.

That’s what have triggered me to do some investigation. First we need to understand what VS is doing when we start debugging in a container with the the Docker profile selected.

VS optimizes the Debug experience by using the so called Fast build mode. In this mode, the docker build command used by VS only uses the base stage of the Dockerfile. Furthermore, when selecting the Docker profile a Project warmup is triggered consisting in the following steps:

Check Docker Desktop is running and is set to the same operating system as project
Pull the image in the first stage (the base stage)
Build the Dockerfile and start the container

When VS starts the container, some volume mappings are created for the remote debugger, and for Nuget folders. You can check the full docker run command in the Containers Tools output window.

The container is now running. We can check it in the Containers window.

Now we want to run our tests inside the container. Basically we just need to run the dotnet test command inside the container. We can do it using the docker exec command.

docker exec -it <name_of_the_container> dotnet test

Let’s see what happens.

We got the following error

It was not possible to find any installed .NET Core SDKs

Now we need to look at the Dockerfile to understand what image is running. This the Dockerfile originally generated by VS.

#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.

FROM mcr.microsoft.com/dotnet/core/runtime:3.1-buster-slim AS base
WORKDIR /app

FROM mcr.microsoft.com/dotnet/core/sdk:3.1-buster AS build
WORKDIR /src
COPY ["XUnitTestContainers/XUnitTestContainers.csproj", "XUnitTestContainers/"]
RUN dotnet restore "XUnitTestContainers/XUnitTestContainers.csproj"
COPY . .
WORKDIR "/src/XUnitTestContainers"
RUN dotnet build "XUnitTestContainers.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "XUnitTestContainers.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "XUnitTestContainers.dll"]

The first thing to notice is the comment at the top of the file. Follow that link to understand the Fast build mode I mentioned earlier. In the Fast mode, only the base stage is used. In this case the base stage is using a runtime image which does not include the .NET SDK required to run tests. We need to change our Dockerfile to use a .NET SDK image in the base stage. We can also remove all the other stages since they are not being used when debugging.

#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.

FROM mcr.microsoft.com/dotnet/core/sdk:3.1-buster AS base
WORKDIR /app

Rebuild the solution. And let’s try again to run the dotnet test command.

Great, we are now running our tests inside the container. Now we just need to have a way to launch the remote debugger and attach to the process responsible to run the tests.

When we run a dotnet test command, a new process is started to run vstest.console to execute the tests, which in turn runs the testhost.dll. We can instruct testhost to pause execution waiting for a debugger to be attached, by using an environment variable VSTEST_HOST_DEBUG=1. Let’s run the same docker exec command passing this environment variable.

docker exec -it -e VSTEST_HOST_DEBUG=1 <name_of_container> dotnet test

We got the following info

_A total of 1 test files matched the specified pattern. Host debugging is

enabled. Please attach debugger to testhost process to continue. Process Id: 643, Name: dotnet_

Now we just need to attach the debugger to the testhost process, which in this case has its PID=643. Let’s use the Container window to attach the debugger to this process.

Wait a few seconds. You may see a screen informing that symbol loading was skipped for testhost.dll.

Just click Continue and your breakpoint is hit.

You can now debug your test inside the Linux container. The whole debugging unit tests experience is not ideal and I truly hope VS integrates Test Explorer with Containers Tools very soon.

Preparing Interviews for a Software Engineer position

Mon, 21 Dec 2020 00:00:00 GMT

Last week I’ve just finished two recruitment processes for a Senior Software Engineer position that I have applied. Both companies have a very well organized recruitment process, consisting in several interviews, including technical ones with code challenges.

I got an offer in both cases. Unfortunately I had to decline one of them, and I can tell you that it was a really hard decision, because both were presenting me exciting and interesting challenges (engineeringly speaking). I wish I could multiply myself to accept both offers (I would have loved to live both experiences). But that’s life. And Yes, I am about to start a new journey. New Year, New Life.

In this post I want to describe how I have prepared for the interviews and give some tips to someone thinking to apply to a similar position. IMHO, preparation is the key to succeed.

Be ready to write code in an interview

The obvious - if we are applying for a software engineer position, we need to be ready to be asked to write code during interviews. I know that some companies do not ask to write code in interviews, they assign you a homework task instead. It’s arguable which one is better.

In my case, in both processes I had to write code (C# and SQL). Furthermore, one of the companies was using a hiring platform, Codility, in which you need to write real working code (compiled, tested against test data, etc.), not just pseudo-code on paper.

The thing is, if you write code on a daily basis, writing code is not an issue, right? The problem here is the type of problems that they are throwing you in the exercises, particularly those that you don’t deal with for some time. They tend to like exercises about:

Arrays challenges
Algorithms and data structures
SQL
Object oriented principles and design patterns

Let’s see what kind of preparation we can think of to improve our ability to solve these problems.

Practicing Code/SQL challenges

There are some platforms out there where we can practice code/sql challenges. Here are some examples:

All of them have great exercises. Regarding difficulty level, I would suggest to focus on the Easy and Medium level. You can choose the language of your choice to solve the exercise.

Codility and HackerRank are hiring platforms, but they have a lot of training content for developers. In Codility, the platform that I used most, I would suggest the 10 first lessons in the section Codility for Programmers. HackerRank is very thorough, and you can choose the skills you want to practice.

And if we struggle in a given exercise, we can have some help from the community using those platforms. It happened sometimes, where I wasn’t tackling the problem in the right direction.

Big-O, Algorithms and Data Structures

This is another topic that engineering interviewers like to discuss or see it in action.

Today, when we need to sort some data, we typically rely on an implementation provided by a base class library. So, if someone asks us out of the blue, what is the difference between a bubble sort and merge sort, unless we have a great memory or deal with this on a daily basis, probably we won’t answer immediately.

But there are some theory that we should know for sure. One of them is the Big O notation (asymptotic analysis). And here, more important than to know what it is, it is to recognize quickly the approx. time complexity of a given algorithm: constant, logarithmic, linear, quadratic, etc.

So, even if we don’t remember the bubble sort algorithm, as soon as we see its implementation we should recognize a quadratic time complexity. Besides sorting, searching algorithms is also important to be aware of, since they are somehow related (we can adopt different searching approaches if the data is already sorted).

A long time ago, I’ve read somewhere a saying: “Make it run, make it right, make it fast”. I can’t remember the source, but I can say that I really like this approach. So, when the interviewer asks you to solve an exercise, focus first on the simplest solution, and do not worry about performance. As soon as you finish your first solution, try to identify its time complexity, and if you have time you can start to identify possible improvements (unnecessary work, common cases. etc.). In the coding platforms that I mentioned earlier, we got a score based on the tests results. Sometimes the solution is correct, but we don’t get 100% score because we are not fast enough, and some tests timeout.

For data structures, I think the important is to know what data structure should be used for a given scenario/exercise. When facing an exercise we should try to identify which data structure, or combination of data structures, best suits the given problem:

Array
Linked List
Queue (FIFO - First in First Out)
Stack (LIFO - Last in First Out)
Hash table
Binary Search Tree
etc.

To know what is the best data structure to use, we need to know the time complexity of each of its operation, such as adding/removing elements, search by position, search by key, etc.

In my case, I reviewed all these concepts by taking a course on Pluralsight.

Object Oriented and Design Patterns

Another topic that interviewers like to ask is about Object Oriented Principles and Design Patterns. And here we can expect several type of interviewers: some of them asks you to explain the SOLID principles; others that ask you to write, for example a singleton pattern; and others that will present you a real problem to see if you use principles and patterns in a natural way. I think this later approach is the one that makes more sense, and in my case it happened exactly that. I was asked to design a system where these topics started to popup naturally. (interfaces, polymorphism, composite pattern, strategy pattern, etc.)

Again, if you use these principles on a daily basis, this should not be a problem. Even some design patterns are so embedded in our daily work that we don’t even notice that we are using them. Nevertheless I decided to review these topics, just in case someone could ask me about them in more detail.

Instead of reading the gang of four book, Design Patterns: Elements of Reusable Object-Oriented Software, I decided to review just the most used patterns by reading the new edition of Head First Design Patterns, which I really enjoyed. The Head First series books provides a great way of learning.

Be ready to brainstorm about real problems

These are the questions that I like most, and in fact I was expecting more of these. Basically the interviewers present you a real problem. Typically you don’t need to write any code. They are just ask you to present a high level solution, discussing alternatives, pros and cons, etc. They want to know how you think.

For example, “the product owner wants a new feature of auto-completing a search, providing multiple matches to the user, based on user past behavior. Let’s start thinking about the problem”. That’s the type of questions you should expect.

IMHO, this is the most interesting type of questions.

Be ready to be asked about past projects

Another common set of questions is questions about past projects that are mentioned in your CV. My suggestion is to revisit your CV and try to remember some options you have chosen, the design and architecture adopted, etc. If some detail is missing in your head, go read some documentation that you still have about the project.

Typically, the projects that are discussed in interviews are chosen by you, not by the interviewer. So, be wise when choosing your projects to be discussed.

There are a lot of topics that can arise during these questions:

DevOps general questions: how you build? how you deploy? What flow you use? CI/CD?
Testing approaches (unit tests, integration tests, e2e test, load tests)
Observability: how do you know what’s happening in production? Telemetry, traces, logs, etc.?
Distributed systems: pub/sub systems? queuing systems? retries? circuit breakers and other patterns, etc.

The variety of questions here is endless, and basically is based on your experience.

Know about the interviewer as much as possible

As I said earlier, preparation is the key to success. In an interview process, part of the preparation is to know the interviewer. Most of the companies announce who will be your interviewer. Take advantage of that information and try to know as much as possible about that person via LinkedIn, blog, presentations, etc.

If we think carefully, interviewing is like dating. Basically both parties are trying to understand how compatible they are, right? If you know who is interviewing you, you can pick common topics that you and your interviewer relates to.

This “investigation” can be very time consuming. Let me give you an example. For the preparation of one of my interviews, I found a presentation made by the interviewer. During that presentation he mentioned that he practices martial arts. Great, I thought, I also practice martial arts (Taekwondo and Hapkido). So we have something in common. At some point in the interview I asked him what martial art did he practice, and we have spent a few minutes on this topic. It’s all about Rapport - connecting through shared interests and empathy.

For another interview, I’ve skimmed the master thesis of the interviewer about Aspects Oriented Programming. I also read his blog. Again, some blog posts were mentioned during the interview. We can create connections easier if we know better who is talking to us.

Soft skills

I am sure that there are a lot of other suggestions to excel in an interview. My focus on this post was mainly in the Software Engineering field. I didn’t want to focus too much in the soft skills. I think the only suggestion that I have regarding soft skills is: be yourself and respect your identity. Don’t pretend to be someone who you aren’t.

Conclusion

As a conclusion, I just want to mention that a recruitment process is time consuming for a candidate. I spent many hours preparing my interviews, and I don’t have any problem assuming that. Probably there are some people that don’t need any kind of preparation. Not me.

In this post I just wanted to describe how I prepared myself for the interviews. It worked for me. It doesn’t mean it will work for you.

Finally I want to finish this post by mentioning two resources:

the book Cracking the Coding Interview. I didn’t read the book so I can’t recommend it. The reviews are great. I only heard about the book after my technical interviews. What I can say is that, if I knew it before, I would have read it before applying to those positions.
the site Glassdoor. Before applying to a position, search for interviews reviews on this site to have an idea what you should expect, how long does it take, how many interviews, etc.

DDD is fancy - Are we doing it right? Should we use it?

Fri, 23 Oct 2020 00:00:00 GMT

TL;DR - This post is just a reflection about DDD based on my experience, and my goal is to reflect about some questions: are we doing DDD right? is it worth? I am far from being an expert in Domain Driven Design (DDD). So, you should skip the entire post if you expect some words from a DDD expert practitioner.

How are we doing DDD

If someone says “My team uses DDD”, that person gets my full attention. Basically I enter in a mode where I can do only two things: to breathe and to get more info from the person in front of me to know more about how they are doing DDD.

Basically, I will get a lot of curiosity to know more about the project, and particularly to know how complex the project is, and what is the layered architecture being adopted. In my opinion, I think it’s dam hard to do DDD right (whatever that means). Furthermore, I will always ask if I can take a look at the code to see what DDD concepts are being used (entities, value objects, aggregate roots, repositories, unit of work, etc.), and what software architecture/patterns are in place (Hexagonal Architecture, Onion Architecture, CQRS, Event Sourcing, etc.). My hope is always to get something new that I can learn from it, or something new that I can say “this makes a lot of sense and I’ve never though about it. I need to try for myself”. Keep learning is always the goal.

Keep studying DDD

I think it was around 2005 when I read the Eric Evan’s book “Domain Driven Design: Tackling Complexity in the Heart of Software”, and I can say that it was very hard to digest it. By that time, my feeling was that there were missing good samples to serve as base reference implementation. It started to appear then some samples in the community, but I never got 100% happy with a single one, where I could say “That is the one”. Even today, from time to time, I still search for new DDD samples to see how they are tackling software complexity.

Fortunately, we have more samples today that we can learn from, at least the tactical aspects of a DDD implementation. I say tactical, because regarding the domain knowledge we don’t have access to the decisions that were made that have conducted to the given model. And that’s where I think we (as developers) are missing one of the most important points of Domain Driven Design: the Ubiquitous Language.

The Ubiquitous Language

I don’t know why, but we (as developers), tend to think that we know more of a given domain than the domain expert itself. It doesn’t make any sense - but we still do it a lot. That’s why there is a gap in the communication between the domain experts and the technical team members. We tend to build something that follows our mental model based on our own technical language, and not based on the domain expert language. We should build based on the Ubiquitous Language.

Unless we are building software that targets developers itself, we are not the domain experts. So we should listen them first, ask them questions to make them think deeper about the subject, and then agree with them on a common vocabulary and model that is understood by all team members in the project. That’s the model that we should build in software. That’s why I think that it makes a lot of sense to have a domain expert as a member of the team. If not, are we really doing DDD? And if we don’t need a domain expert on the team, are we really tackling a complex domain that justifies a DDD investment? Probably not.

Technical complexity

If we are not tackling a complex domain, I think we should focus in not adding technical complexity - instead we should chase for simplicity, technically speaking.

For example, we are asked to build a simple CRUD application - a small backofice web app to manage some reference data. Why should we add technical complexity to this simple app? Probably we can get most of the job done by using a scaffolding tool, combined with a validation, authorization, and auditing mechanism and we are good to go, right? Is there anything wrong with this approach? I don’t think so.

Now, imagine that we are developing a core domain backend service to be consumed by a frontend (a BFF-Backend for Frontend). This core domain is the one that generates money for the company and it’s relatively complex, and it’s evolving quickly.

We decide to use GraphQL to expose our API to be consumed by the frontend, which is mostly a Task based UI. Since we are using GraphQL, we can think that it makes sense to use internally different approaches for queries and mutations, and so we decide to implement a CQRS pattern (Command Query Responsibility Segregation). For the Commands, the rules are complex, and so we decide to adopt a Domain Model approach, with persistence ignorance, repositories, triggering domain events (potentially to be consumed by other services), etc. But for the Queries we decide to use a simpler approach by using a Micro-ORM with an Anemic Read Model (or DTOs if you prefer). Initially, we can even decide that both Queries and Commands are using the same physical database, but we can change that in the future if we need it, by physically isolating the read and write persistence model. Is there anything wrong by mixing these approaches? I don’t think so. Are we really using DDD? Probably we are only using some DDD tactical patterns in parts of our system.

Don’t need to be “DDD All-In”

I think DDD is not an All or Nothing approach. We can use a DDD-Lite (term coined by Vaughn Vernon in his book Implementing Domain-Driven Design) by choosing just some DDD tactical patterns for parts of our system.

I think also that DDD is not for all projects. We all like to put in practice what we learn, and we all want to become better developers writing better software today than we wrote yesterday. DDD is something that I think every developer should learn in the course of his career. But more important is to get the capability to decide when it’s worth or not to use DDD in a project. If we decide to adopt DDD blindly, regardless of the complexity of the domain, just because we are proud of our brilliant code with lots of patterns, I think it’s a wrong approach. At the end of the day we are hired to build software that is supposed to add value to the business. And with this blind approach, we are just throwing technical complexity to the solution, which becomes a cost.

Conclusion

To conclude, I think we should keep learning DDD, to be able to apply it when it worths, and no less important also, to be able to know when it doesn’t worth.

The Fear of Public Speaking

Tue, 06 Oct 2020 00:00:00 GMT

Public Speaking - it’s one of my fears. But I think I am not alone, and according to some studies, it’s a very common fear.

I don’t know exactly the reasons that cause us to be terrified 😨 when we need to speak in public or to an audience - probably there are social or psychological reasons, or even physiological, I don’t know. But I do know how I feel when facing an audience: anxious, nervous, my heart starts beating faster, I start sweating, etc. Typically these feelings have its peak few minutes before the presentation. But then, a few minutes after the presentation starts, everything returns to a normal state, unless there are some issues with the presentation itself, like a laptop crash, or the projector is broken, or anything else that doesn’t follow the plan, and in that case a lot of stress arises. The root causes for these feelings might be: fear to fail publicly, fear to be negativelly commented, fear to be vulnerable, fear to be judged, fear to not know the answer, etc. Today, if someone is on a stage, that person deserves my empathy and my respect - that person is facing these fears fighting them in the Arena (I really like the term Arena, which I heard in the book Daring Greatly).

In 2005 I was invited to be a speaker in TechDays Portugal, an event sponsored by Microsoft, to present a session about the Fallacies of Distributed Computing. I presented the session with a friend and colleague (fortunately I was not alone on the stage) and we decided to present it in a different way: we presented as a parody between two characters. One of the characters was the Teacher, the “Software Architect”, the reasonable, the prudent, and the experienced professional; and the other character was the Student, the “Cowboy Coder”, the developer who thinks is the best in the world and solves everything with a few lines of code. Sounds familiar 🤠? The whole session was presented as a conflict between these 2 characters, with a lot of drama. Guess what character I played:…The fu#$#%&& Cowboy.

Since this session in 2005 got some positive feedback, we were invited again in 2007 to the same event. This time we did two sessions: one about Visual Studio Team Systems and Agile Methodologies (XP, Scrum, etc.), which can be partially watched here; and the other was the continuation of the session of 2005 with the title “The anxieties of an architect”.

For this session we decided to use the same recipe, but now the “Cowboy Coder” became the “Architect Wannabe”, and the Professor was now the Doctor or the Psychologist. Basically the goal of the session was to discuss some questions of that time, such as “XML vs JSON”, or “REST vs WS-*”, “ORM or not ORM”, etc., that were the anxieties of the Architect Wannabe. This time, I played the role of the “Architect wannabe”, and the whole session was designed to be pretended as a psychoanalysis session - I was the whole session laid down in a chaise long, so vulnerable 🛏️. OMG, what have I done? Recently, I found the recording of this session, but I refuse to embed it here in the blog post itself 😄, but you can still watch it here. As far as I remember, the feedback was better in 2005 than in 2007.

My feeling in general is, every time I watch a video of me speaking to an audience, I typically only see the negative points: I think my voice is not good, I think I have the wrong posture, typically my jokes don’t work, etc.

Since then I stayed away of “the stage spotlights”. This year 2020, last February, I was invited to teach a class of Database Management Systems in the School of Business Administration at the Polytechnic Institute of Setúbal, Portugal. This class belongs to the study plan of the 1st cycle college course of IT Systems Management. My role was limited to help students solving problems at the labs, by putting in practice what they were being taught in theory. Basically we are talking about SQL problems, writing a lot of queries, something that I am completely comfortable with.

But then it came the pandemic, and we were all confined to our homes, and most of the sessions were presented online via Microsoft Teams, which is not the same thing as being physically in the classroom. I felt that the level of stress when doing online sessions is much lower.

But since I am going to continue to teach at the same institute, I decided to invest sometime to know and understand how to be a better communicator, and how to improve the connection with your audience. I started watching great speakers and revisiting great presentations. We can find them on TED - go to the list of TED talks and sort presentations by most viewed and we are all presented with excellent speakers and inspiring talks. I confess that I envy them, because they are really great and every talk is absolutely inspiring. I am aware that giving a TED talk is entirely different than teaching a class, but I thought that I could gather something from these TED speakers.

I’ve read the book Talk like TED. The author, Carmine Gallo, studied hundreds of TED Talks and compiled what they have in common in order to be a great presenter. The author presents the 9 public-speaking secrets:

Passion - be passionate about your topic
StoryTelling - tell stories. We tend to pay attention to stories.
Have a conversation - deliver your talk as a conversation with a close friend. Be natural.
Teach something new - present something new to your audience. It can be new information, or just presented in a different and novel way.
Have Jaw-Dropping Moments - deliver shocking, impressive or surprising moments.
Lighten Up - deliver some humor, and don’t take too seriously
18-Minute Rule - keep your presentation length approx. 18 minutes
Multisensory experiences - incorporate several senses in your presentation (sight, sound, touch, smell, and taste)
Be authentic - most people can spot a phony.

I enjoyed reading the book and I strongly recommend it to everyone that wants improve their communication skills. I’ve also annotated each TED talk referenced in the book and I am watching them to observe the subtleties of the speaker.

I would say that is hard do design a presentation with all the secrets in the list, but I will definitely consider them on my next class or talk.

Excluding jobs from Succeeded Jobs in Hangfire Dashboard

Wed, 30 Sep 2020 00:00:00 GMT

TL;DR - If you want to to hide some succeeded jobs from the Hangfire Dashboard, you can do it by forcing them to be deleted as soon as they succeed, and you can use a job filter for that purpose.

Many applications need some background processing, for example, sending thousands of campaign emails. One common scenario is to trigger these background processing tasks in a schedule, for example, “send campaign emails every night at 1 am”.

In the .Net world we have several tools and libraries at our disposal to perform this kind of scheduled background processing. For example, we have Quartz.Net, Coravel, Hangfire, etc.

Recently, I was involved in a project where I needed to build a data connector, a piece of software to perform data synchronization between a 3rd party provider and the customer system, in a multi-tenant fashion environment. Basically, for each tenant the connector needs to run a task to perform the sync operation for the given tenant.

I used Hangfire as the processing background server hosted in a windows service with Topshelf. I won’t go in too much details about the architecture of this connector, but one of the requirements was that it must be reliable and resilient, so as soon as the job was created it should complete the task whenever possible, even if there are some transient errors. Hangfire provides a retry mechanism out of the box, so if an unhandled exception occurs during the job, Hangfire supports to configure a retry strategy to take care of it. I also used some other retries mechanisms before letting the exception to propagate to Hangfire, for example using Polly for HTTP retries.

Another great thing that Hangfire provides is a Dashboard where you can monitor the whole activity in the Hangfire Server, such as, Enqueued Jobs, Processing Jobs, Retries, etc.

Even though the Dashboard feature is great, I also needed to integrate de connector with Azure App Insights, which was the main telemetry system of my customer. One of the things that I was worried about was if the windows service, for some unexpected reason, would stop working. I configured Windows Service to be retried in case of failures, but we never know what could happen, and I was afraid that the stop would become unnoticed.

So, to be 100% sure that I would be notified if the system stopped completely, I decided to run a Heartbeat job every 5 minutes, which basically sends a custom event to App Insights.

And here is the simple code of the job.

public Task Heartbeat() {
    _logger.LogInformation("(.)");
    _telemetryClient.TrackEvent("HEARTBEAT");

    return Task.CompletedTask;
}

With this event being registered in the App Insights, it’s very easy to take advantage of Azure Monitor to create an alert to be triggered if we don’t have a heartbeat event in the last 15 minutes, which means that the service is not running.

So, everything was in place to address my main fear, right? Yes, but I got a collateral effect. Suddenly I got a lot of noise in the Dashboard - every “Heartbeat” jobs was being registered in the “Succeeded” page.

But I didn’t want to pollute the “Succeeded” page with these Heartbeat job entries, and I only care of those jobs related with the data sync. So, I needed to remove them from there. I started to do some research and I eventually found some solutions but they consisted in changing the Dashboard itself, which is something that I wanted to avoid.

So I though, there must be an alternative solution. Hangfire supports some extensibility via Job Filters, which are similar to Filters in ASP.NET. In fact, for this same project, I wrote some filters to manage how the jobs were being enqueued and to manage jobs in queue. So I knew what I could do inside filters.

One thing that occurred to me is that I could change a job to a “Delete” state as soon as it succeeds, and in this case the job would be shown immediately on the “Deleted” page. Here is the simple filter that I eventually wrote.

public class DeleteOnSuccessFilter : JobFilterAttribute, IElectStateFilter {

    public void OnStateElection(ElectStateContext context) {
        if (context.CandidateState.Name == SucceededState.StateName) {
            context.CandidateState = new DeletedState {
                Reason = "Deleted automatically when succeeded."
            };
        }
    }
}

Basically it changes the candidate “Succeeded” state to a “Deleted” state specifying a reason why it was deleted. Next you just need to apply this filter attribute to the job method.

[DeleteOnSuccessFilter]
public Task Heartbeat() {
    _logger.LogInformation("(.)");
    _telemetryClient.TrackEvent("HEARTBEAT");

    return Task.CompletedTask;
}

And here is the result - Succeeded jobs with only the jobs that I am interested in.

And the Deleted jobs with the noise of Heartbeats jobs.

And here is the detail with the reason why it was deleted.

I hope it helps if you have an identical scenario in Hangfire.

From Tumblr to Gatsby

Fri, 27 Sep 2019 00:00:00 GMT

I’ve recently migrated this blog from Tumblr to Gatsby.

I’ve used Tumblr for many years to host this blog. Tumblr can be used for that purpose, but it also provides some other social features like follow, like, reblog, links and video sharing, etc. Moving my blog to a static site generator was in my backlog for a long time, and the main reason is to not have my content dependent of any 3rd party service like Tumblr, Medium, Wordpress, etc. Nobody knows what these services will do in the future with our content. Look for example at Medium, it’s very annoying every time I need to read an article on Medium to be asked to upgrade to a paid plan to continue reading, and forces me to navigate incognito to workaround that. Anyway…

There are many static site generators, and some of them that are very popular (Jekyll, Hugo, etc.). So, the question is, “which one should you use?”. Well, there is no right or wrong answer. I’ve chosen Gatsby, and here are the reasons:

it uses React as templates
it uses GraphQL to query data sources to generate the site
it is written in JavaScript
it has gained a lot of attention recently
it’s easy to learn if you know node and React
it has a lot of plugins to integrate with
it has a lot of starters that you can use as a starting point for your blog, portfolio, company, or e-commerce site.

In summary, I’ve chosen Gatsby because I want to deep my knowledge in the components that uses (React, GraphQL, JavaScript, etc.), and due to its recent popularity in the community.

Blog Migration Requirements

Here is the requirements list for my blog migration:

The content of old posts should be migrated to Markdown files (to be hosted locally in my git repo)
The design should be very simple and clean, allowing me to have an “About” page
It must be able to integrate with Disqus, to allow readers to comment my posts.
Old URLs must be kept (no broken links), including the RSS URL to keep the subscription of my blog in feed readers.
All the typical requirements of a blog about software development: tags, categories, code snippets displayed friendly, google analytics, etc.

Migration of Tumblr content to Markdown

The first step was to get the content of my blog posts as markdown files, including images used in posts. Tumblr provides a REST API that I can use to grab my content. I used a python script, tumblr2markdown.py from someone that had the same need. It worked great, generating a markdown file for each post, and downloading all the images embedded in posts. Each markdown generated contains a YAML front matter with metadata that will be processed by Gatsby. I just needed to do some small changes in the python script to include more metadata, in particular the slug field to be set as the same URL used in Tumblr. (more on that later)

Here is an example with my first post, written in 2012.

---
layout: post
template: post
date: 2012-01-28
title: "Welcome to my blog"
slug: /post/16629534467/welcome
description: "Welcome to my blog"
---
<div>&#13;
<p>One of my resolutions for 2012 was back to blog . I decided to kill <a href="http://www.agilior.pt/blogs/bruno.camara/" target="_blank">my old blog</a> (in fact, it was already dead),  which is hosted in the <a href="http://www.agilior.pt/blogs/" target="_blank">Agilior's corporate blog</a>, the company where I work and I was a co-founder (right now, I already  haven't  any part of the company's capital) and create a new blog with a personal touch,associated with the bfcamara.com domain.</p>&#13;
<p>The most of the content that I will publish is certainly related to my work, software development.</p>&#13;
<p>As one of the founders of Agilior, I hope also to share some of the stories that were part of my life as founder, and some of the mistakes I made along this route. There will be space also to personal subjects, however with lower frequency.</p>&#13;
<p>In <a href="http://www.agilior.pt/blogs/bruno.camara/archive/2007/12/02/3300.aspx">one of my posts</a> at the old blog, I explain that my life has been done of cycles. My perception is that a new cycle is about to come. I do not expect an easy time, and certainly I will have to leave the comfort zone. But "freedom" is one of the things that I value most, and to be honest,  I like to be the leader and the captain of the ship", deciding where to go and when.</p>&#13;
<p><span>I hope I can maintain some activity in this blog.</span></p>&#13;
</div>

As you can see, the content hosted in Tumblr is not very clean, probably because I was using a rich text editor in Tumblr to write my posts, but in the end, what is important is that posts are being displayed correctly in the new blog.

Choose the design by using a Gatsby starter

The next step was to choose the design for my new blog. I wanted a simple and clean design. I started to look to the Gatsby starters library, and I’ve tried some of them.

I ended by choosing the gatsby-v2-start-lumen starter starter. The design is simple, clean, and it’s already prepared to integrate with Disqus. However, in terms of code, I did some changes and I removed some features that I didn’t want to use:

Removed integration with Flow, a static type checker for JavaScript. I thought of using TypeScript instead (also brings optional static type-checking to JavaScript), but in the end I’ve changed the code to be just plain JavaScript (ES2018).
Removed docker files
Changed the Prismjs theme to Okaidia. Prismjs is a syntax highlihter that this Gatsby starter uses to make code snippets embedded in my posts to look great.

Finally, I just needed to set some settings: Avatar, Google Analytics key, Twitter handler, etc, and I had my blog up and running.

Choose the platform to host the blog

Next I needed to choose which platform to use to serve my blog. I had simple requirements: free, support custom domains, and support automatic deployments. By automatic deployments I mean having a new deployment triggered whenever I do a git push in my master branch.

There are many platforms that support JAMstack sites. A JAMstack site (JAM = JavaScript, APIs and Markup) doesn’t need a web/app server. Since I am using Gatsby, which is a static site generator, my blog follows the JAMstack. Here is a list with some platforms that I could have used:

There are many others. I started with Netlify due its popularity, and I must say that the experience was awesome. So, I decided to go with Netlify. But from what I’ve seen from documentation of other platforms, I would risk to say that I wouldn’t have any issue if I had chosen one of them.

In the end, I’ve created my site in Netlify, configured with my custom domain (bfcamara.com), and configured with automatic deployments from my git repo hosted in GitHub.

Keep old URLs (no broken links)

One of my main requirements was to keep the old URLs. Basically I didn’t want to loose all the links out there pointing to my old blog, including the RSS link that people were using to subscribe to my blog.

Part of the problem was solved during content migration. Each markdown created during migration was generated with a front matter key, the slug key, with its value set to the URL used in Tumblr. Gatsby uses this key to determine the URL of a post. For example, for my welcome post I have

slug: /post/16629534467/welcome

The issue is that Tumblr has configured automatically some redirect rules: /post/16629534467 is configured to be redirected to /post/16629534467/welcome. And Tumblr does this for all posts, so the post ID is enough for a post to be visited. I needed to the same in my new blog by using redirects.

Netlify supports redirects, and I just needed to add a new file _redirects to my repo with some rules. For the welcome example above, it’s just as simple as

/post/16629534467 /post/16629534467/welcome

You can check here the file with all the rules.

Finally, I also wanted to keep the RSS subscription URL. In the old blog I was using /rss and the new blog is using /rss.xml. I just needed to add a final redirection rule

/rss /rss.xml

With this rule, RSS readers doesn’t need to change my blog subscription URL.

Conclusion

In conclusion, I am happy with the final result, mainly because now I have full control and I own the content of my blog. I also took advantage of this process to learn Gatsby. I hope it helps anyone that needs to migrate a blog to a static site generator. You can check the git repo in GitHub.

Adopting Gatsby

Fri, 13 Sep 2019 00:00:00 GMT

I finally had some time to move this blog from Tumblr to Gatsby.

I will write a next post describing the whole process of this migration. Stay tuned.

Link post

Wed, 31 Jan 2018 00:00:00 GMT

https://www.simplethread.com/software-complexity-killing-us/

Very good blog post about software complexity nowadays.

Link post

Wed, 20 Dec 2017 00:00:00 GMT

https://blogs.technet.microsoft.com/virtualization/2017/12/19/tar-and-curl-come-to-windows/

It’s about time! Two command-line tools added to the Windows toolchain: curl and bsdtar.

Link post

Tue, 12 Dec 2017 00:00:00 GMT

http://highscalability.com/blog/2017/12/11/netflix-what-happens-when-you-press-play.html

Interesting article explaining how Netflix has evolved its infrasctructure, and how it is today highly distributed all over the world.

Link post

Thu, 19 Oct 2017 00:00:00 GMT

https://www.troyhunt.com/the-6-step-happy-path-to-https/

Excellent blog post explaining how to upgrade a web site to HTTPS safely. Includes also a lot of links to good resources on web apps security.

Link post

Tue, 30 Jun 2015 00:00:00 GMT

https://github.com/blog/2019-how-to-undo-almost-anything-with-git

For future reference, for those git situations we need some help to undo some mistakes.

Link post

Mon, 08 Jun 2015 00:00:00 GMT

http://www.codewrecks.com/blog/index.php/2015/06/03/tfs-new-build-system-vnext-agents

Great post explaining the new TFS/VSO Build system (vNext), particularly the improvements regarding build infrastructure management, when compared with the older approach.

Link post

Tue, 27 Jan 2015 00:00:00 GMT

http://eisenbergeffect.bluespire.com/introducing-aurelia/

I'm totally stoked!!! Why? Because I finally get to share Aurelia with the JavaScript world. What is it you might ask? It's a next generation JavaScript client framework unlike anything else available today. Cut to The Chase Several months ago...

I think we were all expecting this announcement from Rob Eisenberg after he had announced in November last year that he was leaving Angular 2.0. Aurelia is the new JavaScript client framework. I watched the video, and it looks really good. The comparisons to Angular are inevitable, at least for me, because Angular is my framework of choice to develop rich web applications.

I'm very curious to check how the community will embrace Aurelia. I believe that Aurelia is arriving at the right moment after all the controversy about Angular 2.0. It seems obvious that those developers not very happy with the future of Angular 2.0, will embrace Aurelia (or at lest they will try it for sure). Let's start to check Aurelia in Google Search Trends.

The importance of setting goals: half-marathon finished

Mon, 29 Dec 2014 00:00:00 GMT

We are at that time of the year when people typically do a retrospective on how good/bad was the year that is ending. Well, I'm not writing this post to do that kind of reflection, instead I want to show how important is to set goals in our lives.

Last Dec. 7, I finished my first half-marathon, 21 Km (13.1 miles), in 2 hours and 7 minutes.

I would like to write this post telling you how strong I've cut the finish line, but the truth is that it was very hard, particularly the final 3 kilometers. I confess that I spent many minutes thinking to quit, and how absurd I was being in craving for this goal, especially when I don't take any pleasure in running. Let me explain why I did it.

In June of 2012 I did a surgery to my heart. Before the surgery I had strong limitations which didn't allow me to practice sports. With the aorta aneurysm already removed and an aortic prosthesis placed in my heart, a few months later, my surgeon told me that I was ready to do what I want freely except weight lifting. To be honest, I didn't believe him 100%. I decided to make some tests. So, In 2013 I set the following goals for me:

By the end of 2013 I'll run 10 Km. Nice to have: complete it less than an hour
By the end of 2014 I'll run a half-marathon (21 Km). Nice to have: complete it less than 2 hours
By the end of 2015 I'll run a marathon (42 Km). Nice to have: complete it less than 4 hours.

So, I started running in 2013. I was very excited, I bought some shoes, I started to train during the morning, etc. After 2 weeks, I had to stop because my knees couldn't support all this kind of effort (I weighed to much by that time). I stopped for 2 weeks, and then I started gradually. And here are the results:

I achieved the first goal very well: by the end of 2013 I run 10Km in 52 minutes.
The 2nd goal was achieved as well, I run a half-marathon by the end of 2014, however I took 2 hours and 7 minutes to complete it . So, I'd say that I completed the goal.

But when I was cutting the finish line, my first thought was: "hey Bruno, do you think you could handle to run more 21 Kms?". My answer was a big "NO". I have now many doubts if I should go with the 3rd goal at all. It is a lot of pain. It's not just the pain on the day you run a marathon. It's all the pain of the training sessions. Currently. I'm not prepared to run a marathon. In fact I'm wondering if I should drop the goal at all.

The moral of this all story is that if I hadn't set any goal I wouldn't even run 10 KMs. I'm sure about that. This last 3 months, every Sunday, I woke up to run to be prepared to the half-marathon. And the only thought that I had in my mind was "I have to achieve my goal".

On the other hand, it's the goals theirselves that allows us to understand if a goal is realistic or not. And that's why I'm really evaluating if I'll run a marathon or not.

Happy new year 2015.

Video post

Tue, 02 Dec 2014 00:00:00 GMT

Evolving JavaScript with TypeScript: great talk from Anders Hejlsberg where he presents an overview of Typescript.

The recent Microsoft .Net announcements: now what? What's the next complain?

Fri, 14 Nov 2014 00:00:00 GMT

From time to time I discuss with some colleagues, whom I met in university and we graduated together, about the career path in software development that each of us have chosen. I think most of us have chosen the path in an organic way, mostly influenced by the technologies that our first employers had adopted by that time. At least, in my case, it was what happened, and what influenced me the most to have chosen to be a Microsoft .Net developer. However, the majority of my friends have chosen non Microsoft development stacks. Every time we discuss this topic I always have to listen some critics:

".Net is not open source!"
.Net is limited to Windows!
"Developers can't influence the future of .Net"
etc.

Even when I try to explain the Mono project, or how Microsoft encourages developers to participate and vote in UserVoice, the response from them is always the same:

"Mono is a small initiative just to entertain some guys"
"Voting features is not the same as contributing to sources as it happens in OSS projects"

I think most of this behavior of being anti Microsoft have historical reasons. We graduated all in the end of the 90s, and the majority of our projects were developed using the C language targeting Unix systems, with Emacs or vi as the preferred editor. By that time, it was not very fashion to be a Microsoft developer, programming in Basic or C++, COM, MFCs, etc. Therefore, for these guys whom didn't follow the .Net life, it's natural that they still have some issues with Microsoft.

Well, these last two days were made many announcements in the Connect event. To all my those colleagues:

Microsoft open sources .Net
Microsoft .Net will have official distributions for Linux and Mac OS.

So guys, what is your next complain now? I'm looking forward to meet up with all of you to check what are your reactions.

Personal advice to someone starting a software development career

My personal advice to someone that is starting his career in software development is to choose mainly one development stack: Microsoft, Java, Ruby/Rails, PHP, Node, etc. When choosing the stack, don't forget to look at what market is demanding in terms of skills to be sure that what you offer is something that the market needs.

Don't loose sight of what's happening on those different stacks: you'll conclude that typically the trendy practices will be available sooner or later in all development stacks and that the fundamental concepts won't differ too much.

Finally, and especially if you're doing web development where JavaScript is omnipresent, if node is not your main development stack, I suggest to look at it and try to have some knowledge and practice with node (and the frameworks available like expressjs).

It's very hard to follow close what happens in a given development stack, so imagine what needs to be done to excel in various development stacks. But nobody said that it was easy. Everyday there is something new out and it's very hard to know in detail everything. However, be aware that we need to stay close of what is new, otherwise you'll be obsolete soon.

Link post

Fri, 07 Nov 2014 00:00:00 GMT

http://onehungrymind.com/infographic-understanding-angularjs-isolated-scope/

Working with isolated scope in directives can be a bit confusing so I wanted to take this opportunity to try to explain how to interact with isolated scope

Link post

Mon, 27 Oct 2014 00:00:00 GMT

https://docs.google.com/presentation/d/1XQP0_NTzCUcFweauLlkZpbbhNVYbYy156oD—KLmXsk

Check what's coming with Angular 2.

RIP:

Controllers
Directives object definition
$scope
angular.module (no more discussions on how to structure your app in terms of modules)
jqLite

Consuming VS Online REST API using a Service Account

Tue, 07 Oct 2014 00:00:00 GMT

One of the great things about the VS Online REST API is that you no longer need to have installed Visual Studio or Team Explorer, or just the TFS Object Model to integrate with TFS. It's just a normal REST API we can consume by making HTTP calls.

Following the documentation on VS Online site, it seems that we are limited to consume the REST API using alternate credentials (basic authentication with username and password) or using OAuth (bearer access tokens).

This is not perfect if we're developing some integration work in VS Online and we want to use a Service Account to perform some tasks such as queue a build, create work items, etc.

Well, if it's possible to use service accounts when using TFS Object Model, there should be a way to use them as well with VSO REST API . In fact there is a way to use a service account in the REST API, and this post aims to show you how to do it.

First and foremost, we need to get a Service Account in VS Online. For that we can use the tool TFS Service Credential Viewer provided by Martin Hinshelwood from naked ALM. You can even watch a video explaining how to do it.

Next, in order to use these credentials with REST API, we need to understand what TFS Object Model does and try to replicate the same behavior in REST API and check if we succeed. Basically TFS OM uses OAuth WRAP Protocol to get an access token from Azure Access Control Service (ACS).

With the help of Fiddler, here are the interactions we got when using TFS OM:

Hit VS Online unauthenticated to get a redirect 302 response where there are some headers that contain federation data, namely, realm and issuer
Request a token from ACS via the OAuth WRAP Protocol passing the Service Account username and password
Use the returned token in the Authorization header with a WRAP scheme when invoking TFS services.

Let's try to code this in C# and see what we got. If you want a full gist we can grab it from here.

Let's start with Step 1 to get federation data.

Having the federation Realm and Issuer, we can now request an access token

And finally we can use the access token to consumer our REST API, to get the list of team projects for example.

Here is the result for my account

It works!

As a side note, this method of consuming the REST API is not documented in VS Online site which means that we have no guarantees that it will still working in the future.

I hope it helps.

Link post

Mon, 30 Jun 2014 00:00:00 GMT

http://sivers.org/cs

Worth reading. There is no doubt how important customer service is and the impact it has when trying to thrive a company. The same is true for freelancers.

Hubot security considerations

Mon, 23 Jun 2014 00:00:00 GMT

I'm currently working in a project that integrates Hubot with an external system. If you don't know anything about Hubot you can read more here. It's definitely a very endearing project commissioned by Github.

In this small post, I would like to make some security considerations that I thought being useful for someone who needs to install Hubot in production

DIE command

The default Hubot installation (the one that is created by hubot --create mybot) exposes a DIE command, included in the ping.coffee file in the scripts folder. Any user in the room can invoke it as 'hubot DIE'. This command just simply terminates the current hubot instance. A possible mitigation for this risk could be just remove the ping.coffee file from the scripts folder or modify it commenting the command. You could also add an if clause to check if the user running the command has a given role.

Show Storage command

Another command available by default is the 'show storage' command, included in the storage.coffee file. This command shows all the data stored in brain (well, not all data since it's just showing the brain JSON object at a maximum of a depth 4). This command is available for everyone, so be aware of what kind of information you're storing in brain. If you are going to install custom scripts that store sensitive information in brain, you should consider remove this command or add some restrictions on running it. You could also encrypt the data before saving in brain.

Auth vs Roles scripts

The default Hubot installtions loads two scripts to manage roles in hubot

auth.coffee - Only users registered in HUBOT_AUTH_ADMIN env variable can manage roles (kind of secure mode). In this mode only the instance admins can do 'hubot user1 has god role'.
roles.coffee - Everyone in the room can manage roles (kind of a free mode). In this mode everyone can do 'hubot user1 is god'

These modes are mutually exclusive. If env variable HUBOT_AUTH_ADMIN is defined and has values (comma separated list of admin user IDs) then hubot uses the auth script, otherwise hubot uses the roles script. Make sure you define the variable HUBOT_AUTH_ADMIN with the list of user IDs that will be the hubot admins.

HTTP Endpoint secured

You can expose HTTP endpoints inside Hubot which will run hubot commands. Hubot uses internally the Express framework. By default, Hubot exposes these endpoints without any kind of security enabled. We can enable basic authentication by setting the enviroment variables EXPRESS_USER and EXPRESS_PASSWORD. It's strongly recommended to user strong passwords.

Finally I would like to recommend the book Automation and Monitoring with Hubot from Tomas Varaneckas. I bought the book when I started to work with Hubot and I have to say that it helped me a lot to enter quickly inside the world of Hubot.

How much RESTful is your API

Thu, 29 May 2014 00:00:00 GMT

We all have a tendency to say and assume that our API is RESTful as soon as we expose our services using HTTP with JSON/XML and applying some conventions around the HTTP verbs (GET, POST, PUT, DELETE, etc.). However, this assumption is not correct. I don't want to dig in the REST architectural style to show why this assumption is incorrect; instead I want to share something that I learned recently while I'm preparing for my MCSD certification in Web Apps, in particular while studying for exam 70-487: The Richardson Maturity Model.

The Richardson Maturity Model describes four levels of maturity for services

Level 0. Use HTTP as a transport protocol by ignoring the capabilities of HTTP as an application layer protocol. Level 0 services use a single address, also known as endpoint and asingle HTTP method, which is usually POST. SOAP services and other RPC-based protocols are examples of level zero services.
Level 1. Identify resources by using URIs. Each resource in the system has its own URI by which the resource can be accessed.
Level 2. Uses the different HTTP verbs to allow the user to manipulate the resources and create a full API based on resources.
Level 3. Although the first two services only emphasize the suitable use of HTTP semantics, level 3 introduce Hypermedia, an extension of the term Hypertext as a means for a resource to describe their own state in addition to their relation to other resources.

Martin Fowler has a great post in his blog describing in detail these four levels.

I would risk to say that most of us, when using the Web API project template provided in Visual Studio and following the typical recipes, or using the scaffolding generated code, most of the time we are building services at level 2. If we want our services at level 3, we need to introduce Hypermedia elements in our payloads. Maybe we should look to HAL - Hypertext Application Language, which is a simple format that gives a consistent and easy way to hyperlink between resources in our API.

Roy Fielding, the author of the doctoral dissertation that describes REST, considers that level 3 is required in a REST architectural style. However, I think I'll continue to say that "our API is a RESTful API", even though my services are at level 2.

Link post

Thu, 22 May 2014 00:00:00 GMT

http://campus.codeschool.com/courses/shaping-up-with-angular-js/intro

From the video introductions I'd say that it seems an enjoyable way to learn Angularjs.

Using the Zendesk for Visual Studio Online integration

Mon, 12 May 2014 00:00:00 GMT

TechEd NA’s recent announcements about Visual Studio Online (VSO) Service Hooks and REST APIs, will make us believe the sky is the limit regarding integrations between VSO and external services and applications.

The REST API not only allows to get and manipulate data on VSO from a multi-platform perspective but the Service Hooks will enable applications to get notifications about changes on VSO in real time.

But I’m digressing; this isn’t about the APIs nor the service hooks capabilities, but of a particular scenario that has been implemented using these new capabilities.

One of the available service hooks that is available is Zendesk. Using service hooks, paired with a Zendesk app we are now able to reduce the gap between the support team using Zendesk to doing customer care and the development team workflow that is using VSO.

This integration is enabled by using a Zendesk app that allows the support team to better collaborate by escalating tickets to the dev team directly from Zendesk without the need for swivel chair integration.

The app is available as open source in Codeplex’s project the Visual Studio Online App for Zendesk and can be installed on your Zendesk account.

For example, when a customer reports a bug, the agent can create a work item in VSO directly from Zendesk. After fixing the bug, a developer can add a comment to the ticket directly from Visual Studio Online. An agent can also notify developers by adding comments to the work items without leaving Zendesk.

The main purpose of this post is to show how to setup and use the Zendesk for Visual Studio Online integration.

Setup integration

The setup process consists of:

Installing the Zendesk application in Zendesk
Creating a Zendesk service hook subscription in Visual Studio Online

Install application in Zendesk

To install the application you must first download the latest version of the application zip package from CodePlex. You can install the app in Zendesk by navigating to APPS à Manage and clicking in the Upload App button.

Before starting the installation process, Zendesk requires you accept the license. After the upload is done we will be redirected to the settings page to configure the app:

VSO account name à VSO account name, like in https://<account_name>.visualstudio.com.
VSO work item tag à the tag that will be applied in VSO to linked work items. This tag will be used in Visual Studio Online when a work item is created directly from Zendesk.
Enable role restrictions? àselect which roles (administrator, agent) have access to the app

The tag is optional, but by defining a tag you can easily find which work items have been escalated from Zendesk with the use of a query. Furthermore, it also allows you to quickly see on the backlog which work items have been originated from Zendesk (by showing the tags columns)

Fill the settings and click Install.

The app is now installed.

Creating a Zendesk subscription in Visual Studio online

To create a new service hook subscription, in the VSO project portal, we need to go to the a project control panel by clicking the cog icon (any project will do)

In the project’s control panel, choose the tab Service Hooks

Then, click on the link “Create the first subscription for this project”

Start by creating a new service hook subscription by choosing Zendesk as the target service

The Zendesk service hook provides just one action to create a private comment in a Zendesk ticket. This action only supports the event “Work item is commented on”, and before creating a private comment in Zendesk, it performs additional checks:

The comment should start with the pattern defined in the field “Contains only”. When defining a subscription to events “Work item is commented on”, one required parameter is the “Contains string” field.

In other words, only comments that start with the pattern (In this example Zendesk:) will be considered by the subscription. For example

The comment should be related with a Zendesk ticket. The relation is defined using hyperlinks, which means that a given work item is related to a Zendesk ticket if it has in its links a hyperlink to the URL of the ticket details page in Zendesk agent app. (don’t worry this relation is created automatically by the app when the work item is created on VSO, no manual intervention is needed)

When registering the subscription, the following Zendesk inputs will be asked:

The Zendesk account name
User name
API token

Zendesk account name

The Zendesk account name is the subdomain part of the URL used to access the Zendesk account like in https://<account_name>.zendesk.com

Zendesk user name

The Zendesk user name is the email address of the agent that will be used to perform the integration (this can be seen as the integration account which will be used to perform operations on Zendesk). The list of available users can be viewed in People menu option in the Manage group.

API token

The API token is the Zendesk API authentication token for the agent who will perform the integration. To be able to use API tokens a user must explicitly enable access with tokens, which can be done using the API menu option inside the Channels group

We have now a Zendesk service hook subscription created

Using the Zendesk application

The VSO App for Zendesk is only activated in a Ticket side bar. Let’s start by creating a new ticket and submit it.

Go to the details page of the ticket just created and you should see the app in the sidebar. The first time the app is loaded it will ask the agent for his VSO alternate credentials (this gives you full auditability capabilities since the work item will be created in VSO with the agent’s account).

If you don’t know how to enable/set alternate credentials you can read more about it on this Buck Hodges blog post

Fill in the agent Visual Studio Online alternate credentials and click Login.

You should get a success notification.

Since the ticket is not yet associated with a work item, you obviously don’t see the work item, but you can now see the actions that will allow you to create a work item associated with the ticket.

New work item

Now let’s create a work item from a ticket. In the ticket details page, click in New work item button.

The app will open a modal dialog in a loading state while is fetching some data from VSO, and then will show the following form

Fill in the form fields and click Create work item button (the fields should be self-explanatory).

Notice you can choose the work item type; the available work item types depend on the process template where the work item is being created.

The summary is pre filled from the ticket title, but you can change it if you want.

The description is not pre filled, but you can copy the ticket description, by clicking on the link “Copy the ticket’s description”

While the operation is being performed you should see a waiting spinner, and when completed, you will be notified and the work item is now visible as being linked

If you click on the work item title in the sidebar, a new window will be opened in VSO with the work item details. Click on it to the work item in VSO side. You will be able to see how the work item has been created correctly (Id, title, description, work item type, tag and link).

Notice the link to the ticket on the Links tab. This not only allows the Service Hook to determine the association between the work item and the ticket, but it will easily allow the developers to go directly to the ticket Zendesk if they wish too.

Remember, if you delete this link comments made on the work item (using the keyword) will NOT be propagated to Zendesk.

Comment work item in Visual Studio Online

For the work item just created, let’s make a comment in Visual Studio Online

Notice that we started our comment with Zendesk: which means that the subscription will integrate this comment in Zendesk (this was previously configured in the Service Hook configuration, if you used a different pattern then use it accordingly). Go to your Zendesk app and open the ticket, you can now check the new comment is there

View work item details

You can view the work item by click on Eye icon.

This will pop up a modal window that will allow you to see a work item a detailed view without leaving Zendesk

You can customize the fields visible in the summary view and the detailed view by clicking on the Cog icon.

You should see the following page

Here we can choose the fields shown in the summary and the details page. As you start changing your selection, the settings are automatically saved. Let’s add the State to the Summary page and the State and the Severity to Details page.

And now let’s return to the summary page, clicking on the back button.

You should now see the State field

And click again in the eye icon to check the details page and see the State and Severity fields.

To summarize, the summary view is what you get in the sidebar and the detail view is what you view (in a modal window) if you want to see a more detailed view of the work item. This allows you to show the most relevant fields to your agents without the need to ever leave Zendesk

Link to work item

Instead of creating a new work item, we can choose to link the ticket to an existent work item. Click on button Link to work item.

You should see the dialog below

If you know the work item number, just enter it in the Work item text box and click Link. Otherwise you can click Search to execute a VSO work item query.

I will notice you can now see new section called Search that allows you to select the team project and the query to execute (this is the list of queries stored on the team project)

Choose the query you want to execute and click on Query.

Click the Link icon of the row you want to select and its work item id should be copied to the text box. Now you can click the Link button.

While performing the operation you should see a spinner loading, when the operation is completed you should be notified and see the work item in the list of linked work items.

Let’s check how the work item was updated in VSO. As a result of the linking process, the work item should have been updated with a hyperlink with the ticket URL.

Notify

A Zendesk user can send a notification to all the linked work items. This will result in the addition of a comment to all linked work item(s). To notify linked work items click on Notify button.

You should see the following dialog

Fill in the notification and click Notify work items.

We can now check on VSO the new added comment.

Unlink work item

You can also unlink a linked work item, by clicking on the Cross icon.

You need to confirm the operation.

Click ‘Yes, unlink it’ and you should see a spinner while performing the action, you will be notified when the operation is completed.

If you now see the work item in VSO, you will notice the tag and the link to the ticket have been removed.

Conclusion

I believe this is a great example of what possibilities can be achieved with Visual Studio Online REST APIs and service hook integrations.

The Zendesk for Visual Studio Online application is an example of an implementation and a scenario, in which the collaboration between the development team and the support team is increased without going to great hoops, each using their own tools.

I’m really curious what kind of other scenarios people will implement with these new VSO functionalities.

Link post

Sat, 10 May 2014 00:00:00 GMT

http://www.yearofmoo.com/2014/05/how-to-use-ngmessages-in-angularjs.html

ngMessages is a new feature in AngularJS 1.3 for rending error messages in forms

ngMessages is a great new feature in Angular 1.3 which allow us to show/hide/manage form error messages in Angular in a much more elegant way.

Link post

Fri, 09 May 2014 00:00:00 GMT

http://blogs.msdn.com/b/cdndevs/archive/2014/05/06/jump-start-alm-in-your-organization.aspx

Microsoft Virtual Academy courses that will get you started

Send a message to an Azure Service Bus Queue with C# and reading it in Node - Interoperability problems

Mon, 28 Apr 2014 00:00:00 GMT

The other day someone asked me to look at some code in Node. Basically, the code was reading a message from an Azure Service Bus queue. There was nothing wrong with the code, however the person who asked me to help him told me that the message that he was getting from the queue it wasn't exactly equal to the one that was being sent. The message was being written in C#, using the Azure Service Bus SDK. Looking at the C# code, the message was being written with the following code

Nothing special. We are just sending in the message's payload a string representation of the JSON object { 'a': '1'}

And the Node code is reading the message with the following code

When we run the Node code we're getting the following

We are getting an exception when we try to parse the body of the message as JSON because the body of the message we are receiving is

"@\u0006string\b3http://schemas.microsoft.com/2003/10/Serialization/?\u000b{ \"a\": \"1\"}"

And not just what we've sent in C# "{ \"a\": \"1\"}". So, what's happening here?

The problem here is that we are using the BrokeredMessage's constructor that accepts an object as its argument

new BrokeredMessage(payload)

Let's look at its documentation

Initializes a new instance of the BrokeredMessage class from a given object by using DataContractSerializer with a binary XmlDictionaryWriter.

So, the payload is being serialized using DataContractSerializer with a binary XmlDictionaryWriter and this is the cause that explains why the body of the message has in its start the type indication

@\u0006string\b3http://schemas.microsoft.com/2003/10/Serialization/?\u000b

How to fix it?

We have two options to fix the problem:

when reading the message in Node strip the part that we are not interested in
or guarantee that the body of the message is just what we are sending, nothing more

I will focus on the 2nd fix because in my opinion is cleaner. To guarantee tha the body of the message is just what we are sending we have to use other BrokeredMessage's constructor, passing a Stream as an argument

public BrokeredMessage(Stream messageBodyStream, bool ownsStream)

Here what it's said in documentation

Initializes a new instance of the BrokeredMessage class using the supplied stream as its body.

So, in this case we have full control what will be the body of the message. Let's rewrite our C# code to use this constructor

The main difference here is that we are converting our payload string to a stream and use this stream to build our brokered message.

After sending the message to the queue, let's run again the node code

Now we are able to parse the body as JSON and everything is working as expected.

Conclusion: we have to be careful when we have two different worlds in the ends (.Net C# vs Node) and double check for interoperability problems.

Link post

Fri, 25 Apr 2014 00:00:00 GMT

http://peternixey.com/post/83510597580/how-to-be-a-great-software-developer

I believe that your seniority and value as a programmer is measured not in what you know, it’s measured in what you put out. The two are related but definitely not the same. Your value is in how you move your project forward and how you empower your team to do the same.

Worth reading. Great post.

Link post

Tue, 15 Apr 2014 00:00:00 GMT

http://eisenbergeffect.bluespire.com/angular-and-durandal-converge/

Worth reading. Great news in the world of SPAs. Rob Eisenberg, the creator of Durandal, has been working as a member of the AngularJS 2.0 Core Team for almost three months now. They are taking the best ideas of Angular 1.x and the best ideas of Durandal 2.x and converging them in Angular 2.0 to make a truly amazing developer experience.

Link post

Mon, 07 Apr 2014 00:00:00 GMT

https://www.codefellows.org/blogs/complete-list-of-javascript-tools

In our boot camps, students are introduced to several tools and libraries to expand the abilities of their code. Kalina, one of our current JavaScript students, compiled a list of these tools and wanted to share it with other Code Fellows.

A list of javascript tools and resources

Scaffolding
Build
Package management
MVC frameworks
Template engines
Testing frameworks
Backend
Functional programming

Link post

Sun, 30 Mar 2014 00:00:00 GMT

http://sivers.org/tarzan

So my advice is: Change careers like Tarzan.

Don't let go of the old one until the new one is supporting you.

And make sure you don't lose momentum.

As always, great advice from Derek Sivers.

Link post

Fri, 21 Mar 2014 00:00:00 GMT

http://blogs.msdn.com/b/webdev/archive/2014/03/20/test-announcing-rtm-of-asp-net-identity-2-0-0.aspx

Today, we are releasing the final version of ASP.NET Identity 2.0. The main focus in this release was to add security and account management features as well as address feedback from the community.

Great news. Who've worked with version 1.0 of ASP.NET Identity is going to applaud. Great improvements in version 2.0.

Link post

Tue, 18 Mar 2014 00:00:00 GMT

http://highscalability.com/blog/2014/3/17/intuitively-showing-how-to-scale-a-web-application-using-a-c.html

A fun and well written analogy of how to scale web applications using a familiar coffee shop as an example

Great analogy.

Video post

Mon, 17 Mar 2014 00:00:00 GMT

Great way to start my week: watching a great presentation from Scott Hanselman at Fluent Conference 2014. Very good content with a very good performer. Many laughs...

Link post

Sun, 09 Mar 2014 00:00:00 GMT

http://onstartups.com/tabid/3339/bid/105493/Expectations-vs-Reality-8-Lessons-From-The-First-Year-As-CEO.aspx

8 lessons learned from an entrepreneur's first year as a CEO.

IMO, there's an entire generation of entrepreneurs who have a skewed perception that "a great product markets itself." That's bullshit. Sure, there are a few born-viral products like Mailbox or Pebble, but these are the extreme exceptions, not the norms

Worth reading.

Link post

Sat, 01 Mar 2014 00:00:00 GMT

http://fifod.com/an-overview-of-anagularjs-for-managers/

Worth reading. A great analysis of the AngularJS framework, focused on answering the manager's typical questions:

Who is behind Angular?
Is it easy to find Angular developers?
Does Angular supports all browsers?
It is easy to test an Angular app?
Does Angular supports i18n?

Link post

Sat, 01 Mar 2014 00:00:00 GMT

http://taoofcode.net/promise-anti-patterns

Worth reading. If you are using promises, you should read this post. In my case, I'm using some of the anti-patterns mentioned. I'll have to review my promises code.

API's testing tools

Tue, 18 Feb 2014 00:00:00 GMT

I've been doing recently a lot of work integrating different backend systems, using its APIs, tipically in some form of a REST API. The current post has the main goal to enumerate tools that I currently use to become my work easier to explore and diagnose APIs, before getting my hands dirty with code.

RequestBin

I've already introduced this tool in another post. Basically, this tool allows us to check the requests that are being made by our application to a given endpoint.

Postman

Postman is a REST client that runs as a Chrome packaged app. It's an absolutely essential tool to use when you start exploring an API's service provider. Here are the main features:

Create requests quickly:
- supports different formats (json, xml, url parameters, etc.)
- suports basic authentication
- provides oauth helpers
- key/value editors for adding parameters or header values (works for URL parameters too)
Document and share APIs:
- Use collections to organize requests and share them quickly using URLs.
- Document requests inside collections
Collections:
- Collections can be any group of requests.
- You can save any kind of request. While saving the request you can also add a name and a note.
- Collections can be downloaded and shared as a file.

Runscope

Runscope is the company that brings us RequestBin, and it has developed a SaaS app which allows us to test our APIs. It has different components:

Runscope Radar: a test runner and framework
API traffic inspector: allows to check all the traffic that is flowing through our account
Passageway: Passageway allows you and your team to temporarily share your local web server with others via a public URL

For the current work that I'm doing, I've just used API traffic inspector and Passageway

API traffic inpector

It's similiar to Requestbin, since it allows us to capture a given request, but it's more powerfull:

it can act as a gateway to a given real endpoint
it can replay a previous response

Passageway

It was never been easy to expose our local web server to the world. Just download the app and run it.

Zapier

Finally I want to say someting about Zapier. Zapier makes it easy to automate tasks between web apps. Sometimes I have to use Zapier to allow me to test an end to end scenario. Let me give you an example: the system I'm working on provides me an abstraction (a class) that allows me to call a web hook. Internally, the framework that uses that class does not allow to use HTTP calls, and requires that any call would be HTTPS. I was using requestbin to check the payload that the system was producing, but Requestbin does not allow to have HTTPS endpoints (at least without the SSL certificate warning). By that time I didn't know Runscope. So, my solution was to create a Zap in Zapier, that receives on a webhook listening an HTTPS endpoint and sends the request to another webhook on RequestBin. I strongly recommend to check Zapier, since it opens a variety of integration scenarios out of the box.

Conclusion: I'm sure there are other great tools out there, but these are the ones that I'm currently using. Besides the tools I've mentioned,. we should have a good understanding of REST architecture style, and how HTTP protocol works.

Link post

Fri, 14 Feb 2014 00:00:00 GMT

https://docs.google.com/document/d/1XXMvReO8-Awi1EZXAXS4PzDzdNvV6pGcuaF4Q9821Es/pub

A common question that arises among AngularJS developer is "How should I structure my application?". Here are the recommendations published from AngularJS team.

Link post

Wed, 12 Feb 2014 00:00:00 GMT

http://blogs.msdn.com/b/lightswitch/archive/2014/02/11/a-tour-of-the-lightswitch-html-client-apis-elizabeth-maher.aspx

Visual Studio LightSwitch is the simplest way to build modern business applications for the enterprise.

If you want to know the LightSwitch HTML client API, and where you can hook your javascript code, go read it.

Link post

Thu, 06 Feb 2014 00:00:00 GMT

http://www.strathweb.com/2014/02/dynamic-action-return-web-api-2-1/

Dynamically returning IHttpActionResult, HttpResponseMessage or any other custom type from Web API actions

Link post

Thu, 06 Feb 2014 00:00:00 GMT

http://channel9.msdn.com/Shows/Visual-Studio-Toolbox/Unit-Testing-with-Microsoft-Fakes

In this episode, Robert is joined by Carsten Duellmann and Ricardo Wickel, who show us how to improve our unit test using Microsoft Fakes. Fakes help you isolate the code you are testing by replacing

Currently , I'm using Microsoft Fakes, and I have to tell that I'm very happy with it, in special with Shims, which allows me to replace some code by my own implementation and isolate the code I'm testing.

Flatten json object to send within an Azure Hub Notification Template

Fri, 31 Jan 2014 00:00:00 GMT

Have you ever needed to flatten your json to a bunch of properties? If yes, continue to read.

When we need to send a notification to an Azure Notification Hub , we can choose from sending a native notification to Windows, Windows Phone, Apple or Google, or alternatively, use the agnostic method which uses templates.

Currently, I am using the template approach, and here is the method signature

public Task<NotificationOutcome> SendTemplateNotificationAsync(IDictionary<string, string> properties);

So, basically we are only allowed to send key value properties.

Let's say that I have a string representing a json object, and I want to send it using this method. To do this, I need to flatten my json (which can have nested objects and arrays inside it) and get a dictionary with all the keys.

Here is an example of json object

{
	"name": "Bruno",
	"surName": "Camara",
	"address": {
		"street": "My Street",
		"number": 53
		"zipCode": "2840-344"
	},
	"familyMembers": [
		{ "name": "Filipa", relation: "wife" },
		{ "name": "Dekas", relation: "son" },
		{ "name": "Pipo", relation: "son" },		
	]	
}

I want to transform this in a dictionary with the following set of key:value

name:Bruno
surName:Camara
address.street:My Street
address.number:53
address.zipCode:2840-344
familyMembers[0].name: Filipa
familyMembers[0].relation: wife
familyMembers[1].name: Dekas
familyMembers[1].relation: son
familyMembers[2].name: Pipo
familyMembers[2].relation: son

Using the library Json.NET with Linq, it's something that we can do it relatively easy

private static Dictionary<string, string> GetNotificationPropertiesFromMessageContent(string messageContent)
{
	JObject jobject = JObject.Parse(messageContent);

	return jobject.Descendants()
		.Where(j => j.Children().Count() == 0)
		.Aggregate(
			new Dictionary<string, string>(),
			(props, jtoken) =>;
			{
				props.Add(jtoken.Path, jtoken.ToString());
				return props;
			});
}

Basically we are querying for all nodes that are leafs, and in that case we add its path and value to the dictionary.

Here is a console application

And here is the result of running it

I hope it helps.

Link post

Fri, 17 Jan 2014 00:00:00 GMT

http://weblogs.asp.net/scottgu/archive/2014/01/16/windows-azure-staging-publishing-support-for-web-sites-monitoring-improvements-hyper-v-recovery-manager-ga-and-pci-compliance.aspx

One of the other useful Web Site features that we are introducing today is a feature we call “Always On”. When Always On is enabled on a site, Windows Azure will automatically ping your Web Site regularly to ensure that the Web Site is always active and in a warm/running state. This is useful to ensure that a site is always responsive (and that the app domain or worker process has not paged out due to lack of external HTTP requests).

It also useful as a way to keep a Web Site active for scenarios where you want to run background code within it irrespective of whether it is actively processing external HTTP customer requests. We have another new feature we are enabling this week called “Web Jobs” that makes it really easy to now write this background code and run it within a Web Site. I’ll blog more about this feature and how to use it in the next few days.

Great news (from Scott Guthrie's blog). Until now, my typical approach has been to use Quartz.Net to have background jobs, and use a service to ping the web site continuously, using for example SetCronJob. From now on, Web Sites have built-in support for Always On, and very soon will have Web Jobs. Well done Azure.

Running Angular Tests in VS Test Explorer with Chutzpah

Mon, 13 Jan 2014 00:00:00 GMT

The ecosystem around Angular is typically following the Yeoman workflow. This workflow is based on three tools:

Yo - scaffolding tool
Grunt - taskrunner, used to build and test a web app
Bower - dependency management system

I have nothing against this workflow, but since I am a .Net developer, I am used to another tools which reside inside Visual Studio. And I believe that the majority of .Net developers has some reluctance to install node on their machines, which is a requirement installation for all of that tools. We, .Net developers, are accustomed to:

use Project templates to scaffold and bootstrap the app, or Project Item templates with T4 templates to scaffold some specific items
MSBuild as the taskrunner. For running tests we are used to do it inside Visual Studio using the Test Explorer
Nuget as our dependency management system

I’m passionate about Angular. But I’m passionate about Visual Studio either. So, regarding to angular tests, I would like to run them using Test Explorer, instead of grunt. Suppose that I have this angular app inside Visual Studio. It is a very simple app, created from template Empty ASP.NET application, with a Add Unit Tests checkbox checked.

After that, I've added the nuget packages Angular Core and Twitter Bootstrap 3.

Then, I've created an Index.html and an app.js file

Here is the HTML to show a table with a list of tasks

And here is the js code, which includes a controller MainCtrl responsible to return the tasks to the view, using a service dataService which will encapsulate the call to the backend (for now it returns an array hardcoded).

Hit F5 and here is the result

Next, in my test project , I’ve added the packages Angular Core and Jasmine (the test framework).

Then I’ve added my test, which is a normal angular controller test, written in jasmine, asserting that during the initialization of the MainCtrl the scope.tasks is set.

To run the tests in the browser, I’ve added an html file and use the jasmine runner itself to run the tests.

And here is the result when opening this html file in the browser

But my main goal is to run tests using the test explorer, instead of running in the browser.

Introducing Chutzpa Adapter for Test Explorer

Chutzpa ia a javascript test runner which supports QUnit, Jasmine, and Mocha frameworks, and provides a VS Test Adapter. Let’s install it in VS, choosing menu option Tools -> Extensions and Updates

Then, search for Chutzpah, and install extension Chutzpah Test Adapter for the Test Explorer.

After that, restart Visual Studio , and check the Test Explorer Window.

Ok, so now we have our tests in Test Explorer. Let's try to run them.

As you can see, the tests are failing, because Chutzpah is not using the html file to run the tests, hence it is missing some refrences. To fix this, we have to add some comment document references to our test js file

Let's run again our tests.

Voilá. Now I can run my angular tests in Test Explorer.

Link post

Sat, 11 Jan 2014 00:00:00 GMT

http://www.ng-newsletter.com/posts/d3-on-angular.html

Combining the power of D3 and Angular can be challenging and confusing. We constantly get asked how to integrate the two in our classes. In this post, we aim to clear the confusion and bring you the best documentation on how to integrate AngularJS and D3.

A great post explaining how to integrate D3 with Angular.

Link post

Tue, 07 Jan 2014 00:00:00 GMT

http://blogs.msdn.com/b/webdev/archive/2014/01/06/implementing-custom-password-policy-using-asp-net-identity.aspx

To provide a more secure user experience for the application, you might want to customize the password complexity policy. This might include minimum password length, mandatory special characters in the password, disallowing recently used passwords, etc. More information on password policy can be found here. By default ASP.NET Identity enforces a minimum password length of 6 characters. This article provides a simple walkthrough that shows how to add a few more password policies to an application.

Worth reading to understand how extensible is the new ASP.NET Identity System.

Link post

Sun, 29 Dec 2013 00:00:00 GMT

http://paraschopra.com/blog/personal/money-buys-freedom.htm

A life spent mostly hoarding money and possessions is a wasted life

Great blog post. Worth reading.

Link post

Sun, 22 Dec 2013 00:00:00 GMT

http://blogs.msdn.com/b/webdev/archive/2013/12/20/announcing-preview-of-microsoft-aspnet-identity-2-0-0-alpha1.aspx

.NET Web Development and Tools Blog

Do you remember my previous post ASP.NET Identity 1.0: where the hell is the recover password feature? Well, the .Net Web Development and Tools group just announced a preview of Microsoft.AspNet.Identity 2.0.0-alpha. In this new release it is already included Account Confirmation and Reset Password. Well done!

Link post

Wed, 18 Dec 2013 00:00:00 GMT

http://madskristensen.net/post/the-story-behind-_referencesjs

The _references.js file controls a key part of the JavaScript Intellisense in Visual Studio for web projects and can mean the difference between awesome Intellisense and none at all.

Worth reading to help to understand that _references.js fle under the scripts folder in Visual Studio.

Entity Framework: Resetting your migrations to an initial migration

Fri, 13 Dec 2013 00:00:00 GMT

When using Entity Framework Migrations, probably, we are going to have too many migrations, even before we have an initial release of what we are building (I'm not using Automatic Migrations). In one of my previous posts I mentioned this problem around Migrations.Let me show you an example of what I'm talking about.

This is the current list of migrations generated so far for a project I am involved. It's a product that is currently in Beta, and we already have deployed it in a testing environment, where we and our beta testers are performing tests.

I don't want to go into the details of this table, and particularly, how it's used internally by EF, but assume that it is the table that EF uses to know what migrations need to be applied when exists some changes in model.

We are currently at the point where we need to release and prepare a fresh and clean installation in a new environment, but I don't want to have this myriad of migrations, because, in fact, this is my first release and I don't care about the migrations, and how I got this current model, at least in terms of migrations registration. Of course that I do care about knowing how we got the current model version, and for that we have everything registered in source control.

What I really want is:

to exclude all current migrations from the VS project
to generate again an initial create migration with the current model definition
to have, in my new installation, just one entry in table __MigrationHistory
do not compromise an ugrade to the current test environment

1 - Exclude all migrations

This is a no brain step. Exclude all the migrations from the VS project. You just need to have some caution if you have some custom code in migrations, such as for example custom indexes creation. In this case, you need to identify these extra actions which will be used again later.

2 - Generate an Initial Create Migration with all the model definition

Now that we have no migrations in our VS project, we can generate a new migration with all the model definition. First, we need to create an empty database and change the connectionstring in the web.config to use this new empty database.

Then, in Package Manager Console, we generate a new migration called InitialCreate

We should check that the migration generated has all the model definition. You can add to it extra indexes that eventually you had add previously.

3 - Just one entry in the __MigrationHistory

Let's try to update the database running the Update-Database in PM, against the empty database

Then check the __MigrationHistory table

It's ok. Just One entry in __MigrationHistory as we wish.

4 - Do not compromise an upgrade to the current test environment

Now, the interesting part. What happens if you try to update the database against the database that we are using so far, in which all migrations are registered in? Let's change Web.config to reuse again the correct connection string and try to upgrade database to see what happens

Ok. It seems that there is no pending changes, which is true, since I didn't any change to my model. But I want to be sure that the changes from now on are applied correctly. So, let's do a minor change to check how the upgrade behaves.

Basically we are adding a new column 'NewProperty' to an existing table. Let's create the migration

And check the generated migration

So far so good. Now let's check what is the result to upgrade the database, running the Upgrade-Database command, but now I will use the -Script option to generate the script instead of running the upgrade

And here is the script

Ok! The change is applied and the migration is registered. Well done!

Just one more note: from this point you can't downgrade to any migration that was excluded from the project, since the class is not in assembly anymore.

Let's recap

With this recipe you can reset migrations, allowing to have again an initial create migration which reflect the current model, which will be applied as a single migration in a clean installation, and at the same time, do not compromise upgrades on current deployed databases, where migrations are already registered.

ASP.NET Identity 1.0: where the hell is the recover password feature?

Wed, 04 Dec 2013 00:00:00 GMT

Answer: it's not available! Surprised!? Yes, I am very surprised!

For many years we've used the ASP.NET Membership (or the SimpleMembership included in Webpages). Here are the features included out-of-the box in the membership system:

Authentication
User Registration
Roles Management
Password Management
Profile Management

With .Net 4.5.1 we have a new identity management system called ASP.NET Identity

. It's normal to compare the two systems, since they have the same main goal, which is to provide an identity management system.

I don't want to go with the details of this new system, and the reasons that took Microsoft to write it. You can check a great introduction here.

If you decide to use this new system (as I did in my current project) to provide a basic user management system in you application, allowing a user to have an usual username/password credentials, you will be very surprised (and maybe disapointed) by the fact that the password recover feature it's not included out-of-the-box. Of course we can implement it by hand, but the truth is that the ancient Membership system has included this feature, so it seems obvious to think and expect that this new one would also include it. Wrong assumptions. So, if you check the UserManager class (the main class to manage identity features), you won't find any method to generate a reset password token to be used by the application user to be allowed to reset his password.

After some research, it seems that the next version of the ASP.NET Identity, version 1.1, will include the recover password feature.

Since October 2013, Microsoft decided to make available the nightly builds of the ASP.NET Identity system. You can get this nightly builds using NuGet packages available on MyGet. Check the steps to follow in Visual Studio to configure the MyGet feed to pull in the nightly packages for ASP.NET Identity

What I will be showing next is based on the packages with version 1.1.0-alpha1-131108 that I grabbed in beginning of November, which is a pre-release version, and maybe the things have change since then.

Checking again the UserManager class, we now have some methods/properties/interfaces related to the password recover feature

public interface ITokenProvider
{
    string Generate(UserToken token);
    UserToken Validate(string token);
}

public class UserManager<TUser, TKey> : IDisposable
    where TUser : global::Microsoft.AspNet.Identity.IUser
    where TKey : global::System.IEquatable
{
    //...

    public ITokenProvider PasswordResetTokens { get; set; }
    public virtual Task GetPasswordResetTokenAsync(TKey userId);
    public virtual Task ResetPasswordAsync(TKey userId, string token, string newPassword);
    
    //...
}

After taking some time with the disassembled code, what we need is quite simple. We need an implementation of ITokenProvider and set it as the PasswordResetTokens of the UserManager instance that we are using. I am doing this at the AccountController when initializing the UserManager. Here is an example

[RoutePrefix("api/accounts")]
public class AccountController : ApiControllerBase
{
    //...

        AccountController()
    {
        //...
        
        UserManager.PasswordResetTokens = new DataProtectorTokenProvider(Startup.ResetPasswordDataProtectorFactory());
    }
    
    //...
}

After this we can:

use the method GetPasswordResetTokenAsync to get the token to send by email to the user, allowing him to reset the password
validate the token using UserManager.PasswordresetTokens.Validate. In case of success we get an instance of the class UserToken which has the property UserId
and finally, we can use the ResetPasswordAsync to reset the password

So, basically we need an implementation of an ITokenProvider. We can use whatever we want to generate and validate the token, but we can't forget that we are in the middle of a security problem, so it's obvious that we have to be carefull with our implementation. In the example above, I am using the DataProtectorTokenProvider, available in the nuget Microsoft AspNet Identity Owin (the Owin implementation for ASP.NET identity. You can read more about Owin here). This is a generic implementation, which receives an instance of IDataProtector (defined in Microsoft.Owin.Security package)

namespace Microsoft.Owin.Security.DataProtection
{
    // Summary:
    //     Service used to protect and unprotect data
    public interface IDataProtector
    {
        // Summary:
        //     Called to protect user data.
        //
        // Parameters:
        //   userData:
        //     The original data that must be protected
        //
        // Returns:
        //     A different byte array that may be unprotected or altered only by software
        //     that has access to the an identical IDataProtection service.
        byte[] Protect(byte[] userData);
        //
        // Summary:
        //     Called to unprotect user data
        //
        // Parameters:
        //   protectedData:
        //     The byte array returned by a call to Protect on an identical IDataProtection
        //     service.
        //
        // Returns:
        //     The byte array identical to the original userData passed to Protect.
        byte[] Unprotect(byte[] protectedData);
    }
}

In the examle above, I am using the intrastructure provided by Owin Security, which uses Windows Data Protection API. In the Owin Startup class, I define the data protector factory.

public partial class Startup
{
    //...
    internal static Func<IDataProtector> ResetPasswordDataProtectorFactory;

    public void ConfigureAuth(IAppBuilder app)
    {
        //...
    
        ResetPasswordDataProtectorFactory = () => app.CreateDataProtector("ResetPassword");
        
        //...

    }
}

The app.CreateDataProtector uses a DpapiDataProtector.

As a conclusion, I was very surprised by the fact Microsoft decided to not include some features in ASP.NET Identity 1.0, such as password recovery. However, I am very glad to know in advance they are working on a new version 1.1 version, which apparently will include some of the features. For now, you can take the nightly builds to get the features (on your own risk) or implement it by hand.

EF Code-Fist Migrations: How to set the Migrations folder

Fri, 29 Nov 2013 00:00:00 GMT

In my previous post, I confessed that I am currently using Code-First Migrations. If you are using Code-First migrations in your project, at some point in time you've had to enable Migrations, running on the Package Manager (PM) Console the Enable-Migrations command

Enable-Migrations

Are you curious what this command does? Basically, it looks for the current project to find a DbContext, and generates a new class with the name Configuration, which is placed under the folder with the name Migrations in your root, with the following code

namespace Myapp
{
    using System;
    using System.Data.Entity;
    using System.Data.Entity.Migrations;
    using System.Linq;

    internal sealed class Configuration : DbMigrationsConfiguration
    {
        public Configuration()
        {
            AutomaticMigrationsEnabled = false;
        }

        protected override void Seed(MyApp.Data.MyDbContext context)
        {
            //  This method will be called after migrating to the latest version.

            //  You can use the DbSet.AddOrUpdate() helper extension method 
            //  to avoid creating duplicate seed data. E.g.
            //
            //    context.People.AddOrUpdate(
            //      p => p.FullName,
            //      new Person { FullName = "Andrew Peters" },
            //      new Person { FullName = "Brice Lambson" },
            //      new Person { FullName = "Rowan Miller" }
            //    );
            //
        }
    }
}

Every time you want to add a new migration to your project, you have to run in PM console the Add-Migration command

Add-Migration AddColumnFirstName

Here, AddColumnFistName it's the Migration name, and is just an example of a migration.

After this command, you will have a new class 201311291316579_AddColumnFistName added to you project, which is placed under the folder with the name Migrations. The name of the class is generated following the pattern <current_date_time>_<migration_name>.

Below it's an example of the code generated for the migration

namespace MyApp.Migrations
{
    using System;
    using System.Data.Entity.Migrations;
    
    public partial class AddColumnFirstName : DbMigration
    {
        public override void Up()
        {
            AddColumn("dbo.Person", "FirstName", c =&gt; c.String());
        }
        
        public override void Down()
        {
            DropColumn("dbo.Person", "FirstName");
        }
    }
}

I don't like the location of the Migrations folder, which is being placed on the root of my project. Fortunately, you can set this location. For example, I want to place my Migrations in the Data folder. To do this, when enabling migrations, I have to pass the MigrationsDirectory parameter to the Enable-Migrations command

Enable-Migrations -MigrationsDirectory Data\Migrations

Besides of the new location of the Configuration class, there is one more difference in the generated code, in the Configuration constructor

public Configuration()
{
    AutomaticMigrationsEnabled = false;
    MigrationsDirectory = @"Data\Migrations";
}

From now on, every time I add a new migration, this will be added to the Data\Migrations folder. I like to have a clean root project folder.

Just one note: if you already have the Migrations folder in your root, and you wish to have a new location, I suggest to do it manually, moving all the content of the current migrations folder to the new location. Do not foget to change the Configuration constructor as well, setting the MigrationsDirectory property with the new location.

Entity Framework: Database-First vs Code-First

Tue, 26 Nov 2013 00:00:00 GMT

When working with Entity Framework (EF) Code-First, we define our entity model with classes and mappings in code, and the database is generated from the model. When we change the model, we can evolve the database using Migrations.

In the past, I've used more the Database- First approach (rather than the Code First). In the Database-First approach we start to model the database artifacts (tables, views, primary keys, foreign keys, etc.), and then we have the entity model generated from the database.

In the earlier versions, EF didn't support the Code-First approach. Today, EF already supports this approach, and it has been improved in every new version. I decided to try this approach in my current project, and I would like to enumerate some advantages:

all my application "runnable" artifacts are written in C#
no need to have a different database project to accommodate the data artifacts
faster development workflow
makes me think in advance about "the scripts" that will run to upgrade/downgrade my database to accomodate model changes. (every migration has an up and down path, where you can run sql migration statements)

However, I also can enumerate some disadvantages

if your model is changing a lot, and you are working in a team with more than 3 elements, I believe that the team will struggle easily with migrations (the migrations folder starts to grow and it get's out of control)
Some annotation attributes are missing, such as the Index attribute for example. Currently, with EF 6, I cannot decorate my entity properties with an Index annotation. To create an index over a given property, I have to use the fluent API. I tend to prefer the attribute annotations approach.
some lack of control how the database artifacts will be generated

I believe that those more conventional/traditional developers will continue to use the well known Database-First approach. In my case, since I know both approaches, I will support my choice based on context/environment: team and project dimension, type of customer, etc. But now I can say that I'm really confident and confortable with the Code-First approach, and certainly I will use it regularly.

Link post

Mon, 25 Nov 2013 00:00:00 GMT

http://www.codeproject.com/Articles/686808/The-Scams-of-the-Software-Industry

A great definition of what a true software architect is about:

A true software architect can do most of the development by him or herself if required. In short, a true software architect is a hands-on developer with a vision.
A true software architect spends about half of his / her time with the end users and stakeholders. In short, a true software architect is the glue between the end user and the developer(s).
A true software architect is a visionary person that can see ideas into reality. The visions that this architect has is aligned with the company’s goals and executions.
A true software architect has sense of the business needs and can fit himself or herself in the business shoes.
A true software architect can demo the software created before the users and stakeholders.
A true software architect is willing to compromise when there are better solutions available that did not originally come from him or herself. In short, open to suggestions from other technical peers and willing to learn and try out new things.
A true software architect is a master craftsman that has expertise for many technical areas and is versatile with different technologies.
A true software architect is always exploring new ways and is willing to experiment
A true software architect is willing to mentor software developers
A true software architect has the best interest of the company in mind
Ideally, a true software architect is many years experience in different industries because this will allow to discover commonalities across different domains and therefore create solutions that fit.

NLog - How to include custom context variables in all your logs, that will flow across over async points

Fri, 22 Nov 2013 00:00:00 GMT

Update: The Mdlc feature shown in this post is now part of NLog.

TL;DR:If you need to have custom context variables that will flow across async points, you can use the NLog.Contrib nuget package to have the MappedDiagnosticsLogicalContext class and the ${mdlc} layout render.

In my previous post, NLog: How to include custom context variables in all your logs, I presented the NLog.MappedDiagnosticsContext class and how can it be used to emit custom context variables to all of your logs. This class keeps a dictionary of strings in the thread local storage (TLS). When I want to emit always a given context property, I have to add this line of code

NLog.MappedDiagnosticsContext.Set("subscriptionid", SubscriptionId);

In this case the SubscriptionId is the custom context property that I will be attaching to all my log statements. For example, if you are in the context of Web API 2, you can set the property on a base controller, just like this

public override Task<HttpResponseMessage> ExecuteAsync(HttpControllerContext controllerContext, CancellationToken cancellationToken)
{
    if (User != null && User.Identity != null && User.Identity.IsAuthenticated)
        NLog.MappedDiagnosticsContext.Set("subscriptionid", CurrentSubscriptionId);
    else
        NLog.MappedDiagnosticsContext.Set("subscriptionid", null);

    Logger.Info("Begin Request {1} {0}", controllerContext.Request.RequestUri.PathAndQuery, controllerContext.Request.Method);
    return base.ExecuteAsync(controllerContext, cancellationToken);
}

We are overriding the ExecuteAsync method, which is the method that is invoked to start processing the request, and set the subscriptionid property for logging purposes. From this point, all logs will include this new property. My NLog target is defined in configuration as this

<target xsi:type="File" name="f" fileName="${basedir}/App_Data/Logs/App.log"
    layout="${longdate} [T:${threadid}] [L:${uppercase:${level}}] [S:${mdc:item=subscriptionid}] ${message} ${exception:format=tostring}" 
    archiveFileName="${basedir}/App_Data/Logs/Archive/App_${shortdate}.log"
    archiveEvery="Day"
    maxArchiveFiles="30"                
    />

The magic happens using the layout render ${mdc} ${mdc:item=subscriptionid}, which expands the property subscriptionid in the log statement. This is great, since to have the subscriptionid emitted in log files, I don't have to specify the subscriptionid property in all my Logger invocations. Layout renders are macros, and you can know more about layout renders here.

Now let's imagine this code in a ASP.Net Web API 2 controller

[HttpPut, Route("{id:int}/start")]
public async Task<IHttpActionResult> Start(int id) 
{
    Integration integration = await Db.MyIntegrations.FirstOrDefaultAsync(x => x.Id == id);
    if (integration == null) 
    {
        Logger.Warn("The user is trying to start an integration with id={0}, which it doesn't belongs to him", id); 
        return NotFound();
    }  
    
    integration.IsActive = true;
    await Db.SaveChangesAsync();

    Logger.Info("Integration {0} is now active and will start to process", id);

    return Ok();
}

What I want to notice here is, first that I am using async API, and second that I am using Entity Framework 6, which already supports async API, which means that the code in this action will eventually jump from thread to thread while processing the current request. So, the action will start to process in a given thread, and while it is awaiting for the first database call FirstOrDefaultAsync, the thread will be released to the thread pool instead of being blocked awaiting for the database to return. When the database returns, the action will continue to run in another thread from the thread pool. The same process happens when invoking the method SaveChangesAsync. The major advantage of this async paradigm, is scalability, releasing threads that are not doing any job, allowing to use them to process otherrequests. Do not think that your code will run faster by using the async API.

So, the web request will be served using different threads, and this is a problem to my main goal of having the subscriptionid in all my logs. Why? Because the MappedDiagnosticsContext class uses the thread local storage (TLS) to keep the dictionary with the properties, and the TLS does not flow across over async points, which means that when the first database call returns, the thread that will run the continuation it will not have anymore the subscriptionid in the TLS. This behavior relates how .Net deals with ExecutionContext, which is not an easy subject.

To achieve our goal we need to store the dictionary in something that will flow across async points. One possible solution is to use the ancient CallContext. Let's do this in a new class that I will name MappedDiagnosticaLogicalContext

 namespace NLog 
 {
    public static class MappedDiagnosticsLogicalContext
    {
        private const string LogicalContextDictKey = "NLog.MappedDiagnosticsLogicalContext";

        private static IDictionary<string, string> LogicalContextDict
        {
            get
            {
                var dict = CallContext.LogicalGetData(LogicalContextDictKey) as ConcurrentDictionary<string, string>;
                if (dict == null)
                {
                    dict = new ConcurrentDictionary<string, string>();
                    CallContext.LogicalSetData(LogicalContextDictKey, dict);
                }
                return dict;
            }
        }

        public static void Set(string item, string value)
        {
            LogicalContextDict[item] = value;
        }

        public static string Get(string item)
        {
            string s;

            if (!LogicalContextDict.TryGetValue(item, out s))
            {
                s = string.Empty;
            }

            return s;
        }
    }
 }

Now we can rewrite our base controller to use this class instead of the class MappedDiagnosticsContext

NLog.MappedDiagnosticsLogicalContext.Set("subscriptionid", SubscriptionId);

But we also need a custom layout render that can read and expand this items

 namespace NLog.LayoutRenderers
 {
    [LayoutRenderer("mdlc")]
    public class MdlcLayoutRenderer : LayoutRenderer
    {
        [RequiredParameter]
        [DefaultParameter]
        public string Item { get; set; }

        protected override void Append(StringBuilder builder, LogEventInfo logEvent)
        {
            string msg = MappedDiagnosticsLogicalContext.Get(this.Item);
            builder.Append(msg);
        }
    }
 }

This new layout render can be used like this ${mdlc:item=subscriptionid} (mdlc means mapped diagnostics logical context). We need to change our NLog config file, replacing all ocurrences of ${mdc:} by ${mdlc:}.

Finally, and since we are using a custom layout render we have to register it, in the startup of the application

NLog.Config.ConfigurationItemFactory.Default.LayoutRenderers.RegisterDefinition("mdlc", typeof(MdlcLayoutRenderer));

I made a pull request to NLog.Contrib project to incorporate this new feature, and it was already merged and incorporated it in the NuGet NLog.Contrib. So, just grab the Nuget and start using without the need to know the internals.

Link post

Fri, 15 Nov 2013 00:00:00 GMT

http://www.noisli.com/

Noisli is a fantastic background noise and color generator ideal for working and relaxing.

Awesome. An enjoyable idea!!

Link post

Wed, 13 Nov 2013 00:00:00 GMT

http://css-tricks.com/modular-future-web-components/

The following is a guest post by Rob Dodson (@rob_dodson). Rob and I were going back and forth in CodePen support getting Polymer (a web components polyfil

A very good post about Web Components. Do you know what is the Shadow DOM? Worth reading.

NLog: How to include custom context variables in all your logs

Wed, 13 Nov 2013 00:00:00 GMT

Logging is a very important aspect of every application. There are a lot of logging frameworks available for the .Net platform, and I won't discuss which one is better. Many times it is a matter of taste. For example, if we are .Net developers but develop for Java as well, it makes sense to choose Log4Net since it is port of the well known Log4j, so you can capitalize your knowlodge in both platforms.

In the past I've used a lot Log4Net, but nowadays, my logging framework of choice is NLog. For the vast majority of scenarios there is no big difference between platforms, but since NLog was written entirely with .Net in mind, I tend to prefer NLog. Both platforms have great community support and there are a lot of configuration choices available for different scenarios (file logging, sql logging, etc.)

In this post I would like to show you a not very well known feature, in which we can include some custom context variables in our log statements, without the need to specify the variables every time we are emiting the log.

Here is the scenario: we are developing a SaaS web application, and we want to include in all our logs the subscription id, for all web requests.The subscription id identities univocally the subscription that is issuing the web request.

Introducing Mapped Diagnostics Context

From the NLog documentation

Mapped Diagnostics Context is a thread local structure that keeps a dictionary of strings and provides methods to output them in layouts

Here is the code we have write to add a key to this dictionary

NLog.MappedDiagnosticsContext.Set("subscriptionid", subscriptionId.ToString());

After running this line of code, the key subscriptionid is available in the thread local structure. We now need to configure our logging layout to include this key in our logs. Here is a typical configuration

To include the subscription id in the logs we've just used ${mdc:item=subscriptionid}

We can now to grep/find all logs for a given "suspicious" subscription. Let me check some suspicious activity on subscription 25

grep "[S:25]" *.log

When to set Mapped Diagnostics Context

We wish to set our mapped diagnostics context keys as soon as possible. For example, if your application is an Asp.Net MVC application, you can set at the beginning of your action, or as an alternative, in an action filter.

Is this works with async/await?

One thing to notice about MappedDiagnosticsContext is that is a dictionary stored in thread local storage (TLS). This is very important to perceive, in particular if you are in a multi-thread environment. Even if you are using the new paradigm of async and await for asynchronous programing, which is awesome and makes extremely easy for the developer to program in asynchronous way, we have to understand that the TLS does not flow in async points. This means that we will loose this information as soon as we get another thread different from the one where we set the value.

So, how can we make this works with async/await, and guarante that we have always our subscription id in our logs, even if a web request is served by multiple threads during the lifecycle of the request? Basically, we will have to use a different storage from the TLS, one that flows in async poinst. We can use the CallContext. In my next post I will show you how to implement this feature.

Btw, if you are using Log4net, it is already supported out of the box with contexts.

Log4Net supports different scope contexts:

Global (log4net.GlobalContext) - The global context is shared by all threads in the current AppDomain. This context is thread safe for use by multiple threads concurrently. The NLog equivalent is the NLog.GlobalDiagnosticsContext
Thread (log4net.ThreadContext) - The thread context is visible only to the current managed thread. The NLog equivalent is the NLog.MappedDiagnosticsContext.
LogicalTread (log4net.ThreadLogicalContext). The logical thread context is visible to a logical thread. Logical threads can jump from one managed thread to another. For more details see the .NET API System.Runtime.Remoting.Messaging.CallContext.

I haven't figured the equivalent in NLog to the log4net.ThreadLogicalContext. So I had to implement it in my current project. Wait for the next post.

Authentication in a SPA with Angular

Mon, 04 Nov 2013 00:00:00 GMT

There is no doubt that Single Page Applications (SPA) is a trend that is growing more and more each day. However, developing a SPA is harder than the traditional approach, in which the pages are loaded, maybe there is some ajax on the page with jQuery, and when you navigate away to another page a full reload occurs.

When I started my journey developing SPAs, I had to rethink again on some aspects of the application, and check what is the best approach to solve some plumbing problems, such as authentication, authorization, error handling,logging, i18n, etc.

Regarding authentication, I think there are two major approaches:

consider authentication as a separate page, and we only load our SPA, when the user is authenticated
I have a pure SPA, and authentication occurs inside the app, which means there is no need to reload, since the SPA is already loaded.

In both approaches I am assuming and taking as granted that the server always check for access-control (all access control in the client side can be overcome) and that we have an http interceptor that checks the HTTP status response, and if we get an Unauthorized, we redirect the user to the login screen.

In my current application I am working on, I opted for option 2, and I would like to show you how I implemented with Angular, which is my MV* web framework of choice.

Fist of all, I have an Angular service, AuthService, which is responsible to provide helper authentication methods to the app.

(function() {    
    'use strict';

    angular.module('app').factory('authService' , ['$rootScope', 'Restangular', function ($rootScope, Restangular) {

        var user = {
            isAuthenticated: false,
            name: ''
        };

        $rootScope.user = user;

        var authService = {};

        authService.init = function (isAuthenticated, userName) {
            user.isAuthenticated = isAuthenticated;
            user.name = userName;
        };

        authService.isAuthenticated = function () {
            return user.isAuthenticated;
        };

        authService.login = function (loginModel) {

            var loginResult = Restangular.all('accounts').customPOST(loginModel, 'login');

            loginResult.then(function (result) {
                user.isAuthenticated = result.loginOk;
                if (result.loginOk)
                    user.name = loginModel.userName;
            });

            return loginResult;
        };

        authService.logout = function () {
            return Restangular.all('accounts').customPOST(null, 'logout')
            .then(function (result) {
                user.isAuthenticated = false;
                user.name = '';
            });

        };

        authService.register = function (registerModel) {
            return Restangular.all('accounts').customPOST(registerModel, 'register');
        };

        authService.changePassword = function (changePasswordModel) {
            return Restangular.all('accounts').customPUT(changePasswordModel, 'changePassword');
        };

        return authService;
    }]);
})();

My fist implementation consisted of marking routes that need authentication, and detect when one of this routes are in place. The Angular route service provides an event, $routeChangeStart, every time a route changes. Here is the first implementation, that check routes with based on field allowAnonymous

(function () {
    'use strict';

    angular.module('app')
    .constant('APP_ROOT', angular.element('#linkAppRoot').attr('href'))
    .config(['$routeProvider', '$httpProvider', function ($routeProvider, $httpProvider) {

        $routeProvider
            .when('/', { redirectTo: '/integrations' })
            .when('/login', { templateUrl: 'app/views/login.html', controller: 'LoginCtrl', allowAnonymous: true })
            .when('/register', { templateUrl: 'app/views/register.html', controller: 'RegisterCtrl', allowAnonymous: true })
            .when('/integrations', { templateUrl: 'app/views/integrations.html', controller: 'IntegrationsCtrl' })
            .when('/integration/new', { templateUrl: 'app/views/editIntegration.html', controller: 'EditIntegrationCtrl' })
            .when('/integration/:integrationId', { templateUrl: 'app/views/editIntegration.html', controller: 'EditIntegrationCtrl' })
            .when('/activity', { templateUrl: 'app/views/activity.html', controller: 'ActivityCtrl' })
            .when('/changePassword', { templateUrl: 'app/views/changePassword.html', controller: 'ChangePasswordCtrl' })
            //.when('/settings', { templateUrl: 'app/views/settings.html', controller: 'SettingsCtrl' })
            .when('/externalaccounts', { templateUrl: 'app/views/externalAccounts.html', controller: 'ExternalAccountsCtrl' })
            .when('/404', { templateUrl: 'app/views/404.html' })
            .otherwise({ redirectTo: '/404' });

        $httpProvider.interceptors.push('processErrorHttpInterceptor');
    }]);


    angular.module('app').run(['$location', '$rootScope', '$log', 'authService', '$route',
        function ($location, $rootScope, $log, authService, $route) {

            function handleRouteChangeStart(event, next, current) {

                if (!next.allowAnonymous &amp;&amp; !authService.isAuthenticated()) {
                    $log.log('authentication required. redirect to login');

                    var returnUrl = $location.url();
                    $log.log('returnUrl=' + returnUrl);

                    //TODO: BUG -&gt; THIS IS NOT PREVENTING THE CURRENT ROUTE
                    //This has a side effect, which is load of the controller/view configured to the route
                    //The redirect to login occurs later.
                    //Possible solutions: 
                    // 1 - use $locationChangeStart (it is hard to get the route being used)
                    // 2 - Use a resolver in controller, returning a promise, and reject when needs auth
                    event.preventDefault();

                    $location.path('/login').search({ returnUrl: returnUrl })

                }
            }

            $rootScope.$on('$routeChangeStart', handleRouteChangeStart);

        }]);
})();

However, there is a problem with this solution, since the code e.preventDefault() it does nothing. Basically the route is not prevented, which means that the destination controller is loaded, and eventually some API calls to the server are being done. In functional terms the user is redirected to the login screen, however we can have API calls that are not necessary due the fact that the controller is instantiated.

So, the correct approach consists to prevent the route to succeed. How can we do that in Angular? Well, in Angular, we can participate in the resolving process before the route succeeds. Here is the Angular documentation for the when() method of $routeProvider:

resolve - {Object.<string, function>=} - An optional map of dependencies which should be injected into the controller. If any of these dependencies are promises, the router will wait for them all to be resolved or one to be rejected before the controller is instantiated. If all the promises are resolved successfully, the values of the resolved promises are injected and $routeChangeSuccess event is fired. If any of the promises are rejected the $routeChangeError event is fired.

So, if we have a promise that is rejected when the user tries to achieve a route that needs authentication and the user is not authenticated, then the route is aborted and the event $routeChangeError is fired.

Based on this fact, here is the final implementation

(function () {
    'use strict';

    angular.module('app')
    .constant('APP_ROOT', angular.element('#linkAppRoot').attr('href'))
    .config(['$routeProvider', '$httpProvider', '$locationProvider', function ($routeProvider, $httpProvider, $locationProvider) {

        function checkLoggedIn($q, $log, authService) {
            var deferred = $q.defer();

            if (!authService.isAuthenticated()) {
                $log.log('authentication required. redirect to login');
                deferred.reject({ needsAuthentication: true });
            } else {
                deferred.resolve();
            }

            return deferred.promise;
        }

        $routeProvider.whenAuthenticated = function (path, route) {
            route.resolve = route.resolve || {};
            angular.extend(route.resolve, { isLoggedIn: ['$q', '$log', 'authService', checkLoggedIn] });
            return $routeProvider.when(path, route);
        }


        $routeProvider
            .when('/', { redirectTo: '/integrations' })
            .when('/login', { templateUrl: 'app/views/login.html', controller: 'LoginCtrl' })
            .when('/register', { templateUrl: 'app/views/register.html', controller: 'RegisterCtrl' })
            .whenAuthenticated('/integrations', { templateUrl: 'app/views/integrations.html', controller: 'IntegrationsCtrl' })
            .whenAuthenticated('/integration/new/:integrationType', { templateUrl: 'app/views/editIntegration.html', controller: 'EditIntegrationCtrl' })
            .whenAuthenticated('/integration/:integrationType/:integrationId', { templateUrl: 'app/views/editIntegration.html', controller: 'EditIntegrationCtrl' })
            .whenAuthenticated('/activity', { templateUrl: 'app/views/activity.html', controller: 'ActivityCtrl' })
            .whenAuthenticated('/changePassword', { templateUrl: 'app/views/changePassword.html', controller: 'ChangePasswordCtrl' })
            //.whenAuthenticated('/settings', { templateUrl: 'app/views/settings.html', controller: 'SettingsCtrl' })
            .whenAuthenticated('/externalaccounts', { templateUrl: 'app/views/externalAccounts.html', controller: 'ExternalAccountsCtrl' })
            .when('/404', { templateUrl: 'app/views/404.html', controller: 'NotFoundErrorCtrl' })
            .when('/apierror', {templateUrl: 'app/views/apierror.html', controller: 'ApiErrorCtrl' })
            .otherwise({ redirectTo: '/404' });

        $httpProvider.interceptors.push('processErrorHttpInterceptor');
    }]);

    angular.module('app').run(['$location', '$rootScope', '$log', 'authService', '$route',
        function ($location, $rootScope, $log, authService, $route,) {

            $rootScope.$on('$routeChangeError', function (ev, current, previous, rejection) {
                if (rejection &amp;&amp; rejection.needsAuthentication === true) {
                    var returnUrl = $location.url();
                    $log.log('returnUrl=' + returnUrl);
                    $location.path('/login').search({ returnUrl: returnUrl });
                }
            });

        }]);

})();

Basically the method whenAuthenticated adds the checkLoggedIn resolver to the route resolve object, which asks to the authService if the current user is authenticated. If not, it rejects the promise, and the $routeChangeError is raised, which in turn is handled redirecting to login route.

I hope it helps

Link post

Fri, 01 Nov 2013 00:00:00 GMT

http://chrisguillebeau.com/3x5/risk-something-to-gain-something/

“If you only do things where you know the answer in advance, your company goes away.” -Jeff Bezos

Link post

Tue, 22 Oct 2013 00:00:00 GMT

http://onstartups.com/tabid/3339/bid/102728/How-To-Launch-A-Startup-Without-Writing-Code.aspx

A startup should be about solving a customer problem -- which does not always mean building a product from scratch.

Link post

Fri, 20 Sep 2013 00:00:00 GMT

http://wintellect.com/blogs/jlikness/10-reasons-web-developers-should-learn-angularjs

An excelent post describing the the top 10 reasons why we should learn and use AngularJS. It is an extensive post, well written. Worth read. Here is the summary

AngularJS gives XAML developers a place to go on the web: the principles reflected in AngularJS allows us to have the same advantages of the XAML frameworks (WPF, Silverlight), such as data-binding, parallel work between designers and developers, great separation of UI and the logic that fetches and processes data
AngularJS gets rid of ritual and ceremony: just plain javascript objects to participate in the data-bindng process. No need to extend from an existing object.
AngularJS handles dependencies: just tell angular what you need and angular goes anf gets it for you. It's all about dependency injection.
AngularJS allows developers to express UI declaratively and reduce side effects: The UI is well structured, and by declaring your UI and placing markup directly in HTML, you keep de presentation logic in one place and separated from the imperative logic. You can even extend HTML using angular directives.
AngularJS embraces DD...Er, Testing: AngularJS helps you to write tests easy. The framework is prepared to unit testes or end-2-end tests
AngularJS enables parallel development: AngularJS defines a set of concepts (modules, controllers, services, directives, etc.) which help us to structure our application in a way that allows us to have easily multiple developers working simultaneously on different parts of the application
AngularJS enables a Design <-> Development workflow: in AngularJS it's more easy to designer add markup without completely breaking an application, due the separation between the UI and logic.
AngularJS gives developers controls: AngularJS has a concept named directives which allows to create new HTML elements and attributes. Basically it's a way to increase the browser vocabulary
AngularJS helps developers manage client state
AngularJS supports Singe Page Applications: AngularJS provides all the necessary foundations (routing, templates, views, history, etc) to build a SPA

Link post

Thu, 19 Sep 2013 00:00:00 GMT

http://wildermuth.com/2013/09/18/Getting_Ready_for_AngularJS_1_2

Link post

Thu, 05 Sep 2013 00:00:00 GMT

http://www.uibox.in/

UIBox - it seems to me a very good initiative. Almost any Web UI component that we can think of has already been built by someone.

A search on Google results in hundreds of results. All this is great!

Except for a few problems: no demos, poor documentation, dependencies conflicts, browser support, etc.

The UIBox initiative tries to help solving this problems. To start with, UIBox is a curated HTML, CSS, JS UI component library. There are a few requirements to a given component be present in UIBox:

must have a working demo page
must be well documented

You can check the motivations and reasons for the UIBox initiative.

Link post

Thu, 29 Aug 2013 00:00:00 GMT

http://highscalability.com/blog/2013/8/28/sean-hulls-20-biggest-bottlenecks-that-reduce-and-slow-down.html

This article is a lightly edited version of 20 Obstacles to Scalability by Sean Hull ( with ...

A great article enumerating the biggest bottenecks thar impact scalability:

2-phase commit
Insufficient or no caching
Bad Disks, Raid 5, Multitenant storage
Serial processing
Missing feature flags (enable/disable features)
Single copy of database
Using database for queuing
Using database for full-text searching
Object Relational Mappers (ORMs)
Missing instrumentation
Not using a version control system
Single points of faillure
Lack of browse-only mode
Weak communication between team members
Lack of documentation
Lack of fire drills
Insufficient monitoring and metrics
Cowboy operations
Growing technical debt
Insufficiente Logging

Leaving Agilior....

Mon, 05 Aug 2013 00:00:00 GMT

Last week was my last working week at Agilior. Nine and half years after I have founded Agilior, at 4 January 2004, today it ends a cycle in my life. As most of you already know, because you know me personally or have read my post "My First Big Mistake as a Founder", I already have not any shares of the company, and currently I was just an employee.

This way out was expected for some time due the fact that Agilior, in the present moment, could not give me the professional achievement that I need. Hence I'm feeling with a mix of feelings of happiness and nostalgy. Happiness because this way out it is something that I already wanted some time ago, but I have not had the courage to do it. Nostalgy because I look back and I remember great moments that I lived at Agilior, in particular the times that I helped to grow a company which started with 3 people, becoming in a company with 9 people. I've had a great pleasure to work with these people, building software development projects.

So, I'm finishing now a stage in my life, and in this moment I need some time to think and reflect about what to do next, and decide what way to follow. I will take off some vacations on August with total dedication to the family, and avoiding being online.

Switching off

LightSwitch and the Enterprise

Tue, 16 Jul 2013 00:00:00 GMT

After reading this post about LightSwitch in the Enterprise, I decided to give my two cents.

I started developing with LightSwitch since the first Beta version, which was lauched in 2010, in last days of August. At that time, I was developing with my team a web application around Governance, Risk Management, and Complicance (GRC). We were using a typical web stack with ASP.Net MVC, Entity Framework (I can't remember if it was Linq2Sql at that time), Html, Javascript, in particular jQuery. The truth is that our velocity was very slow, and we were constantly struggling with the user interface. As soon as the first Beta 1 of LightSwitch was available to download, I decided to give it a try. Since we had the data model already designed, I tried to use LightSwitch to develop the majority of the screens that we had, but now using Lightswitch. The final result was awesome, since I spent no more than two days to build the screens that had been developed in a month. The application was now a silverlight app running on the browser, with a different look and feel, but you know what: the product owner liked the "aesthetics" of this new silverlight app and we decided to move on with LightSwitch to build the first version of app (which was designed following the Minimum Viable Prodcut approach, to test the idea of the product).

Since then, I have already used Lightswitch in many other situations, developing applications that are currently running in production. I don't have doubts that Lightswitch has a corner in the enterprise applications space, allowing a development team has a boost in productivity, and seeing their development costs reduced.

I am happy to watch Microsoft investing in Lightswitch, improving, for example, the collaborative development on Lightswitch applications, which was already announced to be present in Visual Studio 2013.

Even though it seems I am LightSwitch fan, I am not saying that LightSwitch is a silver bullet, and that we should forget everything we learned so far. Far from that. What I am saying, is that LightSwitch can be used in the Enterprise without any fear. I suggest to start with small applications, maybe to serve a few department users, and specially if you have a lot CRUD screens (a typical small backoffice application, perhaps). I think you will be surprised how fast you develop this kind of apps with Lightswitch.

I am not saying also that Lightswitch does not have any problems. As any other technology, Lightswitch can present some problems. For example, you have to know very well the technology to avoid some performance issues you can have. LightSwitch advocates a kind of development, and you have to know it well. When we are constantly struggling with LightSwitch to do things, perhaps it is a signal that we shouldn't be using LightSwitch. There are a lot extensibility points there, but it's not comparable with the full control you have when using the tools you want (for example, Asp.Net MVC, WebApi, HTML5, CSS, jQuery, etc.).

As a conclusion, I just want to say that Lightswitch has a space in the enterprise. Of course every organization is different, with different requirements, and above all, with different believings when regarding software development, which can produce some resistance to adopt this kind of RAD tool. Give it a try to use Lightswitch in a small CRUD line-of-business application, and in an economical point of view you will be surprised about the results.

Link post

Mon, 01 Jul 2013 00:00:00 GMT

http://blog.brunoscopelliti.com/a-directive-to-manage-file-upload-in-an-angularjs-application

This week I was in the need to add the functionality of file upload to an AngularJS web application, on which Iâm working for a personal project. Since my project also uses jQuery, I found that the...

Link post

Mon, 24 Jun 2013 00:00:00 GMT

http://www.yearofmoo.com/2013/01/full-spectrum-testing-with-angularjs-and-karma.html

Learn how to fully test your AngularJS application with Karma in every possible area using E2E, Unit and Midway testing

Link post

Tue, 18 Jun 2013 00:00:00 GMT

http://amitgharat.wordpress.com/2013/06/08/the-hitchhikers-guide-to-the-directive/

AngularJS Directive is what we have close to Web Components at this time and is the most crucial and difficult piece in AngularJS. I've not found any suitable article on the web to fully understand...

Running a desktop app with AngularJS served from a Windows Service with Katana (OWIN)

Tue, 04 Jun 2013 00:00:00 GMT

Nowadays, when developing desktop applications for Windows, we have many options when choosing the .Net technology to use:

Windows Prsentation Foundation (WPF) - in my opinion, this is the most consensus choide
Windows Forms
Silverlight

If the target is Windows 8 we can add even more choices to the pile, for example using HTML5 and Javascript, due the support of the new WinRT architecture.

Recently I had to analyze a scenario where a windows service should be developed, which would be responsible to run some kind of scheduled tasks - polling a SaaS service on the public internet, and integrate some data with on-premises systems. Additionally, a graphical tool should also be developed, to allow to configure some aspects of the windows service (instead of editing configuration files by hand).

For the UI tool, my first approach was to choose a WPF application. But I have no experience at all with WPF, and as a developer I am more comfortable with the web stack development, which means that on the client side I would like to use Html5 and Javascript, and the UI tool would be running on the browser.

Since I'm in love with AngularJS, my wish was to develop the tool using AngularJS. But I don't want to complicate the installation process, requiring my tool to be hosted in Internet Information Service (IIS). So, what I really want is to install my window service, which besides the integration logic, should also host my AngularJS application.

Well, in fact, what I really need is to have some kind of embedded web server that supports:

Static files - (html, images, css and javascript files)
AspNet Web Api - I want to use WebApi to return the data to the graphical tool

With this goal in mind, it's time to meet OWIN (Open web Interface for .Net) specification and the Katana Project.

From the OWIN site

OWIN defines a standard interface between .NET web servers and web applications. The goal of the OWIN interface is to decouple server and application, encourage the development of simple modules for .NET web development, and, by being an open standard, stimulate the open source ecosystem of .NET web development tools.

So, basically, OWIN is a spec which allows components to be reused on different web servers.

The Katana Project is a collection of projects to support OWIN with various Microsoft components. It is also a command line application for running self-host servers. To have a great overview of the Katana project read this post.

Ok, stop theory, and show me the how can I use Katana to embed a small web server on my windows service. which allows me to build an AngularJS app. Here we go!

First, let's create a new Visual Studio project, choosing the type Windows Service

Next, let's make some modifications to the project to allow us to test without having to install the windows service:

Change the project type to Console Application (Project Properties)

Change the Start and Stop methods of the windows service to allow calls from a console app

Add a conditional compilation statement #if to allow to run in Debug as a console application

Now let's start to embed our web server. We will need some nuget packages

The first NuGet we will need it's the Microsoft.Owin.Hosting, which allow us to have a basci OWIN hosting in our application

Notice that the package is a pre-release version, so we should change the option in the drop down "Stable Only" to "Include Prerelease". To create our web application host, we just need to modify out internal start and call the method WebApplication.Start, indating the listen url as parameter (http://localhost:12345).

The Start method is a generic method, where we have to tell what is the class responsible to do the initial startup, which in our case it is the WebStartup class. By convention, the method to be called it is named Configuration, receiving the application builder object as an argument

Let's try to run our application, hitting F5, to see what happens

We are getting an exception, telling us that the HTTP Listener is not found.

Could not load file or assembly 'Microsoft.Owin.Host.HttpListener' or one of its dependencies. The system cannot find the file specified.

Until now, what we did was just adding the hosting capabilies to our application. We need to specify the OWIN component that should listen for and serve HTTP requests. Let's add the Microsoft.Owin.Host.HttpListener NuGet, which is the OWIN component that uses the Microsoft Windows HTTP protocol stack (http.sys)

Let's hit F5 again and see what happens

As you can see nothing happens, and the exception disapears. However I want to have a quick way to check if the hosting is correct. To do this, we will add another NuGet Microsoft.Owin.Diagnostics, which will allows to have a quick diagnostics

Next, we need to tell to our web application to use the diagnostics capabilities, calling the method UseTestPage() during the configuration phase

Let's hit F5 again, and open a browser and type our url on the address bar

Cool! Now we have an http listener which serves always this simple test page, greeting "Welcome to Katana". Now we need to add another NugGt, Microsoft.Owin.StaticFiles, which will allow us to serve our static files (Html, CSS, Javascript files, etc.). For some reason, this nuget is not being returned in the search result when using the window "Manage NuGet Package". So, we will have to use the Package Manager Console and use the command

Install-Package Microsoft.Owin.StaticFiles -Version 0.20-alpha-20220-88 -Pre

Now we need to configure our static files component to tell the path where the static files are. In my case I choose to create a folder webdir to be the static files container, and it will be located in the same directory of the executing assembly

Let's add an Index.html file to webdir folder and try to navigate to this page in the browser (do not forget to mark this file to be added to the output directory)

Hit F5 and browse to http://localhost:12345

Very good. Now we can serve static files. Half of our problem is solved, since an AngularJS application it's just a bunch of Html, javascript, CSS, and imagesfiles.

But if you remember, we need also to use WebApi, which is the way to return some data to the app. Let's add another NuGet, Microsoft.AspNet.WebApi.Owin, which will allow us to host WebApi controllers and provide a REST api on our application.

Now we need to configure the WebApi, telling the route pattern to use.

Basically we configured to use the route /api/{controller} tou our WebApi controllers. Let's add a simple HomeController with the Get method, returning a simple string.

Let's Hit F5 and browse to the url http://localhost:12345/api/home

The WebApi content-negotiation resulted in a json fle. Let's click Open to see the content

Bingo. Our application now handles WebApi.

Finally, to install this as a windows service, we just need to add a project installer

Compile in release mode, install the windows service (using installutil.exe) and you are ready to add your AngularJS application, which will be served by the windows service process, without the need of Internet Information Service (IIS). With this, now I can consider to build my configuration tool (to run on desktop) using AngularJS and WebApi.

You can download the full code in my github account.

Link post

Fri, 31 May 2013 00:00:00 GMT

http://www.asp.net/vnext/overview/owin-and-katana/an-overview-of-project-katana

The ASP.NET Framework has been around for over ten years, and the platform has enabled the development of countless Web sites and services. As Web application development strategies have evolved, t...

A very good white paper explaining OWIN and the Katana Project. It begins to be very easy to have a lightweight web server embedded in our own process (console applications, windows services, etc). It opens a lot of possibilities. I'm dirtying my hands, trying to host an AngularJS application inside a Windows Service.

Video post

Wed, 22 May 2013 00:00:00 GMT

False

The AngularJS video tutorial course featured on Egghead.io

Video post

Fri, 17 May 2013 00:00:00 GMT

I'm in love with AngularJS. Here is the presentation about design decisions on Google I/O 2013.

It is impressive how the Google feedback project, with 17K lines of code turned into a project with 1.5K lines of code using AngularJS.

Link post

Fri, 10 May 2013 00:00:00 GMT

http://www.infoq.com/news/2013/05/nobackend

At the Front-Trends 2013 conference last week, Gregor Martynus gave a talk entitled "Look ma, no backend!" about developing applications primarily from a front-end perspective, falling back to using server-side components only to implement the features the browser does not yet support.

Link post

Wed, 08 May 2013 00:00:00 GMT

http://unicornfree.com/2013/why-you-should-do-a-tiny-product-first

A good post from Amy Hoy, where she advices that we should make the first product an inforproduct (e-book, screencast, workshop, etc.), like 37signals did. This advice goes along the line of thought of the quote that

the best marketing is education

ASP.NET MVC - Default Model Binder with HTML Validation

Mon, 15 Apr 2013 00:00:00 GMT

This post demonstrates how we can handle gracefully the HTTP request validation which exists in ASP.NET, when using ASP.NET MVC. Request validation is a feature in ASP.NET that examines an HTTP request and determines whether it contains potentially dangerous content. For example, if a user tries to input some malicious code in a field using the script element, the ASP.NET throws a "potentially dangerous value was detected" error and stops page processing. Without this validation, the site is vulnerable to this exploit typically referred as a cross-site scripting (XSS) attack.

Consider this form

Here is the error when the user tries to submit a field with HTML elements, pressing the Save button

We can disable this validation at a field, action, controller, or application level. But we have to think about the consequences of disabling it and being vulnerable to an XSS attack. Instead of disabling it, I want to continue validating the potential dangerous requests, but rather than throw the error and stop processing, I want to add a validation message for the field that is violating the validation.

In ASP.NET MVC, we can do this using a custom default model binder. The model binding is the mechanism that automatically populates the controller action parameters, taking care of the details of property mappings and type conversions. The default model binder in ASP.NET MVC is the class DefaultModelBinder . The good news is that we can have a custom default model binder. In our case, we want to inherit the DefaultModelBinder, overriding the main method BindModel, and handle the exception HttpRequestValidationException which is thrown when a potential dangerous request is detected

public class DefaultModelBinderWithHtmlValidation : DefaultModelBinder
{
    public override object BindModel(ControllerContext controllerContext, ModelBindingContext bindingContext)
    {
        try
        {
            return base.BindModel(controllerContext, bindingContext);
        }
        catch (HttpRequestValidationException)
        {
            Trace.TraceWarning("Ilegal characters were found in field {0}", bindingContext.ModelMetadata.DisplayName ?? bindingContext.ModelName);
            bindingContext.ModelState.AddModelError(bindingContext.ModelName, string.Format("Ilegal characters were found in field {0}", bindingContext.ModelMetadata.DisplayName ?? bindingContext.ModelName));
        }

        //Cast the value provider to an IUnvalidatedValueProvider, which allows to skip validation
        IUnvalidatedValueProvider provider = bindingContext.ValueProvider as IUnvalidatedValueProvider;
        if (provider == null) return null;

        //Get the attempted value, skiping the validation
        var result = provider.GetValue(bindingContext.ModelName, skipValidation: true);
        Debug.Assert(result != null, "result is null");

        return result.AttemptedValue;
    }
}

So, what we do is quite simple: we catch the HttpRequestValidationException exception and add a validation error message to the ModelState specifying the field name that is violating the request. Next we read the attempted value, skipping the validation and return. In our action controller we can't forget to check if the ModelState is valid.

[HttpPost]
public ActionResult Create(MyModel model, string fieldArgument)
{

    if (ModelState.IsValid)
    {
        //The model is OK. We can do whatever we want to do with the model
        model.MyMessage = "Model Ok Updated @ " + DateTime.Now;
    }

    ViewBag.FieldArgument = fieldArgument;
    return View(model);
}

Next we have to register our binder as the default binder. We do this in the application start

//Register our binder as the default model binder
ModelBinders.Binders.DefaultBinder = new DefaultModelBinderWithHtmlValidation();

You can check the full source code here on my github.

Link post

Mon, 18 Mar 2013 00:00:00 GMT

http://swombat.com/2013/3/15/taking-the-leap

A very good post from Daniel Tenner on his successful startup blog, swombat.com, where he gives some advices to those who want to go to “taking the leap” (quit your job and start your own company). Here is the list, with my personal comments from my experience of co-founding a company and primarily from my mistakes

“Cut (personal) costs to the minimum” - In the begining it is absolutely essential to cut the personal costs, since there is no income (worst case scenario, but a realistic one). There are some expenses that can’t be cut. Most of the trouble here is caused by decisions made in the past: big house, big car, etc. If you are married it is better to inform your wife about the potential changes of this movement: go out to dinner less often, postpone some house improvements, etc. And it is also absolutely crucial to answer to the question “how many months can you handle without any income and what is your stop loss limit?”. if you have a partner, do not skip this question and have an answer from your partner as well. It does not make sense that you accept to handle no income during 12 months and your partner just can handle 2 months.

“Do not take funding” - I believe that there are some kind of businesses where it is needed to take some funding. Although, I think it is always preferable to be self-funding. First, if you take money from some VC or if you sell part of the company, you maybe loose the operational control, and some frictions arise as consequence. If you follow the self-funding path, as Daniel refers, you will be pressured to make sales and get some revenue. This pressure will force you to be very pragmatic and force you to validate your idea as soon as possible. You will be forced to launch early, which in my opinion it is a good thing. Even if the product does not gain any traction, you didn't waste all your time and money. If we have to fail, then we should fail early, allowing us to have more second chances.

“Connect with mentors and peers and listen to them” - Absolutely right. I think I should have listened more people in the field, who already took the leap. In 2004, when I decided to found my company, there was too much wishful thinking around my head. But that's it: "wishful thinking", with some distance from the reality. Listening experiences from other people, and knowing in advance how hard it is to take the leap it is crucial to handle hard times.

Don't tie yourself to one idea - It is very difficult to know if our idea it will be or not a successful idea . I think the execution is more important than the idea, however picking the wrong idea, even with a good execution, can be something that it's not profitable. The Lean Startup approach popularized from Eric Ries (credits here also to Steven Blank with the Customer Development techniques popularized in the book the "4 Steps to the Epiphany"), can help us to rapidly validate our idea. If the feedback of this validation it's not what we are really expecting, we have to be prepared to pivot or jump to another idea. But the lack of focus on an idea can be bad to the execution. Derek Sivers, from CDBaby tell us that the Ideas are just multipliers of Executions. My observations on this field tell me that a possible recipe, with apparently some kind of success, consists in building something to an audience that you know well, and the audience recognizes you as an authority and trusts you. The hard part is to grab the audience. But there are a lot of guys that follow this recipe with some success: start a blog which gains buzz, write a book, record screencasts, develop workshops, etc. Then finally build a product to this audience. However, this evidence of success doesn't tell us nothing about the efficacy of this recipe, since it is not possible to know how many people have tried this recipe and have failed.

"Don’t make plans or set deadlines" - Well, I don't agree with this statement. I think the statement should be "Don't make a detailed plan or set rigid deadlines". I think that we should have a plan an write it on piece of paper, primarily if you are co-founding with a partner. But the plan should be no longer than a page. "The shorter your plan the better", very well said in this post . But Daniel gives an excellent advice: if you for some reason have to rollback, then get a daily-job in a startup instead of a large company. By that time, you should be known in the startup scene, and maybe you go-back for a job without the need to be interviewed. Very good advice.

SQLFiddle - sharing database problems and their solutions

Mon, 04 Mar 2013 00:00:00 GMT

If you are a web developer, probably you know jsFiddle. If you don't know you can always start play with it today. jsFiddle it's an online playground tool where you can test your javascript/css/html problems and share it with others. Many StackOverflow answers, the answers site for developers, include a link to jsFiddle to demonstrate a problem and/or a solution. jsFiddle became very popular among web developers. There are other online playground tools, but they are tipically focused in just one technology, HTML or CSS or Javascript. jsFiddle combines the three.

Until recently, I didn't know any tool similar with jsFiddle, but focused on database problems. But there is! It's the SQLFiddle. And if you use jsFiddle you start to recognize the similarities, and in seconds you start to use SQLFiddle without problems.

The other day I was talking with a guy, who started testing my SQL knowledge and asked me to solve a problem.

Here is the problem:

If you have a table with a date column and an int column, write a query which gives me the interval start date and end date, and the sum of the values for the interval. In other words, if we have these records in table

Dt1, 5
Dt2, 6
Dt3, 2
Dt4, 8

I want the result

Dt1, Dt2, 11 (5+6)
Dt2, Dt3, 8 (6 + 2)
Dt3, Dt4 10 (2 + 8)

The problem was not new for me, since I already had to face it in real life. In this new version of Sql Server 2012 we have to functions Lead and Lag that we can use to solve the problem without having to think too much. So my first answer was "Can I assume that I am using a Sql Server 2012?" The guy is a MySql guy, so he answered me "No. Write a query that we can port to another database without too much effort". So, I wrote the query on paper, which consists of a self join combined with the use of row_number() function.

The guy looked to the query and didn't recognize the row_number() function. Then he show me a query using his approach with MySql.

None of us had our laptop, prepared with the tools to show the solution to each other using a real database instance. We could use SQLFiddle to show the solution to each other.

BTW, here is the solution without using Lead and Lag

Link post

Mon, 18 Feb 2013 00:00:00 GMT

http://unicornfree.com/2013/a-customer-is-your-mvp-a-video-talk-on-making-products-that-sell

Great session from Amy Hoy, where she talks on making products that sell. Her advice: don't look for an idea. Look for an audience; understand and know your audience; kill a pain of your audience with a unique selling proposition (USP).

Link post

Sat, 02 Feb 2013 00:00:00 GMT

http://onstartups.com/tabid/3339/bid/94858/The-4-Personality-Types-Every-Startup-Needs.aspx

A great post from Dharmesh Shah, on OnStartups, where he identifies the 4 most important startup personalities to build an unbeatable team:

The Beast (The X-men), who has a "get shid done" mentality
Lara Croft, with an adventurer and entrepreneurial spirit
The Architect (the character from the Matrix), who understands the big picture and can still focus on the details.
The Most Interesting Man in the World, with character, charm, and adds depth to the company culture.

Worth reading.

WebHooks testing tools: Requestb.in and LocalTunnel

Mon, 28 Jan 2013 00:00:00 GMT

The goal of this post is to show how we can use the tools Requestb.in and LocalTunnel to make it easier the process of testing WebHooks integration.

WebHooks offer an easy and elegant way for applications and services to integrate with one another. There is a trend that we have been watching in the last years, which consists in having an interconnected set of loosely coupled cloud services, talking to each other via HTTP. The term "webhooks" was introduced by Jeff Lindsay in 2006, as a pattern in web architecture. Both services, RequestBin and LocalTunnel were created by Jeff.

Today there are a lot of cloud services which already offer WebHook integration, such as Zapier, UserVoice, ZenDesk, GitHub, etc.

The idea is quite simple. Taking the UserVoice as an example, when a new Ticket is posted on my UserVoice, I want to trigger an HTTP call to my CRM system to create a case and record the customer interaction on it. This is a typical use case of a webhook integration.

Here is the screen of user voice where I am defining the WebHook integration

A WebHook is just an HTTP callback sent to a URL, defined by the user in response to an event.

When I started to develop this kind of webhook integrations I felt the need to have a quick way to have real examples of the POST messages. The first Cloud service that I used to have this reals messages is Requestb.in.

RequestBin lets me create a URL that will collect requests made to it, for further inspection in a human-friendly way. With RequestBin you can look at webhook requests. Here as an example of a request made in response to a new suggestion on the UserVoice site.

With Requestb.in you have a quick way to inspect the HTTP request that you have to handle. But what I really like is to have a way to test the WebHook in my local machine, which means that I have to expose my web server to the world, which in some scenarios it's not easy (firewall, NATs, etc.). We can use an easy way with LocalTunnel to solve this problem.

To use localtunnel, I need to install it in my local machine. Following the instructions on site, I rapidly noticed that I need Ruby and RubyGems, which I don't have it installed. Alternatively, and if I use version 2, I need to have Python, since the client is written in Python.

You can just skip this installation and use localtunnel for windows, which is written in C#, and available on GitHub. That's what I did. In a further post I will show the how to use LocalTunnel on IIS Express.

Maybe a not well known GO statement in Sql Server Management Studio

Tue, 15 Jan 2013 00:00:00 GMT

Recently it was launched a new blog targeted to the community involved with Microsoft Sql Server (www.sql.pt). The blog was launched by a friend of mine, Pedro Correia, which is a Sql Server 2008 Microsoft Certified Master (I think it's the only one in Portugal).

Well, first of all let me say that I consider myself a guy which is well above the average in terms of Sql Server and T-SQL. Since the beginning of my career, which was in 1999, I had to deal with Sql Server and I have been involved in projects that I had to understand the internals and write all the T-SQL by hand.

I was reading a blog post on this new blog when I saw this snippet of code

CREATE TABLE T1 (C1 INT);
GO
INSERT t1 SELECT CAST(RAND() * 100 AS INT);
GO 27
INSERT t1 VALUES (310),(450),(945);

At first glance, there is one thing in this script that surprised me a lot:

GO 27

My first reaction was, this script doesn't run, since the syntax it's not valid. I copied and pasted to my Management Studio, with a connection to a Sql Server 2012, and Bang! It runs! Basically it runs 27 times the insert that precedes this GO statement.

Oh my god: how I couldn't know this? I checked the documentation of the GO statement and here what I read

GO [count]

count Is a positive integer. The batch preceding GO will execute the specified number of times.

I felt so frustrated because I didn't know about this optional argument that I started to check the documentation of previous versions. Basically I was trying to justify my ignorance. The Sql Server 2005 already supported this syntax. Got dammit! However, the documentation available for Sql Server 2000 in here does not reference any optional count argument. Well, I don't have a chance to test a Sql Server 2000 right now, but I believe that this is something that was introduced with the Sql Server 2005. I started my career with Sql Server 7...bla..bla..bla... No excuses! I am an ignorant! But I didn't remember to be presented to this feature.

The fact is that this optional count argumenf of GO statement it's useful, particular when testing some situations in Management Studio, when we need to load some data for the tests being run. We can always use a while statement to do some repetitions, although this GO [count] it's much more elegant.

As a final remark, the GO is not a Transact-SQL statement; it is a command recognized by the sqlcmd and osql utilities and SQL Server Management Studio Code editor.

Link post

Thu, 20 Dec 2012 00:00:00 GMT

http://blog.bahadir.io/posts/failed-entrepreneur.html

Great post about the reasons why Bahadir thinks he had failed as an entrepreneur. Every item of this list it's a lesson to all of us, who wishes to follow this path.

The Mini Sales Site approach as the first MVP when developing a SaaS application

Tue, 11 Dec 2012 00:00:00 GMT

A minimum viable product (MVP) is a core concept of the Lean Startup movement. The MVP is that version of the product that enables a team to test and validate the fundamental hypotheses (the leap-of-faith assumptions) and help entrepreneurs begin the learning process as quickly as possible, allowing a full turn of the Build-Measure-Learn loop with a minimum amount of effort and the least amount of development time. But when the time arrives to get your your hands dirty, it remains the question: How minimum is the minimum in MVP? It's not an easy question to answer.

We know that the MVP will lack many features that may prove essential later on. And since we are trying to validate our hypotheses and measure the impact of the product, we will need to build something that we can measure from. And the first metric that we need to know is: do we have buyers for our product?

In my continuous learning process about startups and how to bootstrap, I read the book "Start Small, Stay Small" from Rob Walling, which I strongly recommend for those who pursuit the Micropreneur lifestyle. In this book I learned a new approach, which seems to me to fit in the MVP concept boundaries: The Mini Sales Site. The main idea of this approach is to reduce uncertainty about the conversion rate, and whether or not the prospect would buy our product.

Asking to a potential customer "You will buy my product?" in a survey, or even building a Landing page asking for an email in case he is interested in the product, it's seems not enough and provide us with inaccurate data.

The Mini Sales Site is a stripped version of what will be the Sales Site. This approach will force us to think how to market our product, even before we start coding. A few years ago I thought that the most important think to get the path to success it was the quality of the product itself. Don't get me wrong: a product with quality it's important (although quality in the context of software development, among developers, it's usually associated how clean is the code, and how beautiful patterns were used in the solution, etc., which it's not a correct definition btw), but marketing is more important in my opinion.

The Mini Sales Site consists of 2 or 3 pages:

Home
Tour (optional)
Pricing and Sign-up

The Home and Tour page will enumerate the benefits (better to enumerate the benefits rather the features) of using the product, screen shots and/or videos, the full product information, and most important a call to action to try the button 'Try Now' button.

The Pricing and Sign-up page will contain the prices and the different editions availabe of the SaaS application, with call to action buttons "Try Now" or "Select Plan".

Next we create an AdWords Campaign, configured with our keywords, and send the targeted traffic to our Mini Sales Site.

When someone clicks on "Try Now" or "Select Plan', we will notify the prospect that the product is under development yet and will be available soon, asking for the email address to notify him when the product will be available.

We should also have configured the AdWord conversions, to know which keyword was converted and the money spent on each conversion.

Of course I have mixed feelings about this approach, since it seems that I am fooling my prospect. But in fact, the most people on the web are browsing, and if the user it's not interested in the product, he will bounce, leaving the mini sales immediately . If they are interested, maybe they don't mind to sign up for a launch notification.

At the end of the day, with this approach you can measure the searches performed with the keywords defined in AdWords, which keywords converted, and have a rough estimate of the conversion rate. We get this with a small investment, including the AdWord campaign, the development and design of the HTML pages of the mini sales site, and the screen-shots of the application if we have it.

In summary, I would like to say that the Mini Sales Site it seems to me a valid approach for those that who desire to build an MVP for SaaS applications, without much investment involved. It can be the first iteration of the Build-Measure-Learn loop, without compromising the next iterations.

EF5: Entity Framework cache problems

Mon, 03 Dec 2012 00:00:00 GMT

The majority of ORMs supports a first-level cache (L1), common named as "identity cache". Basically, if we request the same entity twice by its ID, it doesn't require a new round-trip to the database. EF5 is no exception, and provide us with the method Find() on the DbSet<T> class. From the documentation we have that Find

Uses the primary key value to attempt to find an entity tracked by the context. If the entity is not in the context then a query will be executed and evaluated against the data in the data source, and null is returned if the entity is not found in the context or in the data source. Note that the Find also returns entities that have been added to the context but have not yet been saved to the database.

Given that, if I make the call

db.Contacts.Find(id);

I know what is the expected behavior. The main problem exists with this code below

//Get the contact for Id=1
var contact = db.Contacts.First(x => x.ContactId == 1)
Console.WriteLine("The Current Name is: " + contact.Name);

//Update the contact in the same session without using the context tracking
db.Database.ExecuteSqlCommand("update dbo.Contacts set Name = 'Filipe' where ContactId = 1");

//Get the contact again
var updatedContact = db.Contacts.First(x => x.ContactId == 1);
Console.WriteLine("The Updated Name is: " + updatedContact.Name);

Assuming that the initial name is André, what do you think is the output of this code? Here is the output

To be honest, this is not what I expected to see. I was expecting to see André followed by Filipe.

After looking at the code, I thought: well EF is smart enough to see that I am using the same query twice,

db.Contacts.First(x => x.ContactId == 1)

with the same params, in the same session, and is caching the data, then the 2nd call does not go to the database. Cool, I thought.

Let's confirm with the SQL profiler, expecting to see just one query

I was totally wrong. The EF goes to database twice, getting the contact by Id. However, EF does not refresh the entity that is in context. Maybe this a a by design decision, to not sacrifice performance, since it would be required to merge the entity that is currently in context with the entity fetched from the database. However, in my opinion, it's hard to accept the fact that EF goes to the store to fetch the entity, and does nothing with it if the entity already exists in context.

To get the updated entity in context, we have two options :

1) detach the entity before querying it again

((IObjectContextAdapter)db).ObjectContext.Detach(contact);

2) Refresh the entity

((IObjectContextAdapter)db).ObjectContext.Refresh(System.Data.Objects.RefreshMode.StoreWins, contact);

If you opt for the Refresh option, it is not necessary to query again. Here is the final code

//get the contact
var contact = db.Contacts.First(x => x.ContactId == 1)
Console.WriteLine("The Current Name is: " + contact.Name);

//update using ExecuteSqlCommand
db.Database.ExecuteSqlCommand("update dbo.Contacts set Name = 'Filipe' where ContactId = 1");

//Refresh the entity
((IObjectContextAdapter)db).ObjectContext.Refresh(System.Data.Objects.RefreshMode.StoreWins, contact);
Console.WriteLine("The Updated Name is: " + contact.Name);

Here is the ouput

This a warning to all readers that can have this situation in a project, for example, when in the same context execute a stored procedure that modify entities that are already loaded in the current EF context, and for some reason you need to refresh the entities. Do not assume that by seeing the statement in the SQL profiler that EF will merge the data in context.

Samsara on Software Development

Mon, 26 Nov 2012 00:00:00 GMT

One of the books that touched me deeply was "Siddhartha", a novel written by Herman Hesse . I wish I had been aware of this book earlier in my life. What motivates me to write this post, is a concept that this book has introduced me into: Samsara.

Briefly, at some point of his life, around 40, Siddhartha believed he had fallen into a state of Samsara, which in Sanskrit (the primary liturgical language of Hinduism) means ignorance and confusion. Siddhartha, an extremely spiritual person in his childhood, was becoming an ordinary person in a gradual way. He became materialistic and ostentatious, practicing vices like drinking, gambling, etc.. He was obfuscating and denying his spirituality with mundane activities. Throughout all these years of "success" with signs of great wealth, the life of Siddhartha was in fact shrinking in terms of meaning. One day, Siddhartha gets tired of his life, one that he consiedred as empty and meaningless, and decides to return to a path of spirituality. It's this state of confusion that Siddhartha defines as Samsara.

In terms of software development, I believe we (as an industry) are also in a Samsara. At least for me, sometimes I am really confused with all the technologies I can use, and with the speed that things changes. We, as developers, love technology, and tend to wish to experience the ultimate edge technology in our project. The market players themselves, such as companies that sell software for developers or open source communities, send us updates and new products in very short time cycles. And we, as lovers of technology, are continually pursuing them. I understand that this is a way to influence the the market in terms of supply, which otherwise would stagnate. But the real question I want to ask is: are we really producing better software than 20 years ago? All this volatility and “wealth” in our development toolbox allows us to actually produce better software? Maybe not! Eventually we are producing software that is more complex, but is this all complexity really necessary?

I think that we have lost the north by amusing ourselves with technological onanistic practices, which don’t contribute to the improvement of software development. Undoubtedly its really cool to use X or Y, and at the end of day we can eventually add a few more skills on LinkedIn profile. “Yeah, I am really geek!” But the thing is, why do we write software? Yes, Why we do write software? We (should) write software to achieve a particular goal, typically the resolution of a problem for those who will use it. And one thing is certain, if someone pays us to write software, that same person wishes to get their money back (Return on Investment) in the form of value. Typically, he/she doesn't care about the technology stack used to developed it. Instead, we should be concerned in solving their problem(s), and ensuring that what we produce has a low maintenance cost, and a quick resolution time on error situations. With this in mind, I agree with Jack Dorsey, founder of Twitter and Square, who wrote a few weeks ago a post “Let’s reconsider our users”, where he mentions

“ all of our work is in service of our customers. Period.”

He goes further, and he decide to abandon the term ‘User’, replacing it everywhere in Square by the term “Customer”.

Many times,we throw more technology into to the solution, just because, or just because the community says so, or because I'm a fantastic geek. The truth is: this doesn’t solves anything, zero, nada, zilch. Worse: it's one more technology that will have to be maintained and learned.

And with all this technological Samsara, we are dazzled, and lose focus from our goal: to solve the customer problem.

But do not worry, I'm sure that one more month, some company will release a new IDE or a new Framework to the market that will solve all of our problems.....

Let's get more focused, and nominate yourself as a "Problem Solver".

Link post

Tue, 06 Nov 2012 00:00:00 GMT

http://blog.folyo.me/the-startups-guide-to-budget-design/

How-to: 0-legged OAuth with DotNetOpenAuth

Tue, 30 Oct 2012 00:00:00 GMT

As promissed in my previous post, where I dived in the the difference between a 2-legged and a 0-lgeed oauth, I will show in this post how to build a 0-legged oauth using the DotNetOpenAuth library. Just to remember, a 0-legged oauth consists of a consumer accessing the protected resources submitting oauth signed requests using just the consumer key and the consumer secret.

I will take as an example the UserVoice API, the method to list all the users for a given site, which requires a 0-legged API.

Before you use the UserVoice API or another oauth API from a service provider, the first thing to to is to get a consumer key and a consumer secret. This process is more or less the same in all service providers, but you have to check with the service provider how to get them. Here is a screenshot how I got this data from UserVoice site, in the admin console

Next, in our visual studio project, we have to get the libraries of dotnetopenauth. Since we are consuming an oauth API, we have to get the Nuget package DotNetOpenAuth.OAuth.Consumer.

The main class when consuming an API is the class WebConsumer in namespace DotNetOpenAuth.OAuth. The constructor of this class receives two arguments as show below

public WebConsumer(ServiceProviderDescription serviceDescription, IConsumerTokenManager tokenManager);

The serviceDescription defines the endpoints and the behavior of the service provider. The token manager is the component responsible to manage the tokens used by a web site in its role as a consumer.

In our case the serviceDescription is something like this

var providerDesc = new ServiceProviderDescription()
{
	RequestTokenEndpoint = new MessageReceivingEndpoint("https://bfcamara.uservoice.com/oauth/request_token", HttpDeliveryMethods.PostRequest),
	AccessTokenEndpoint = new MessageReceivingEndpoint("https://bfcamara.uservoice.com/oauth/access_token", HttpDeliveryMethods.PostRequest),
    UserAuthorizationEndpoint = new MessageReceivingEndpoint("https://bfcamara.uservoice.com/oauth/authorize", HttpDeliveryMethods.PostRequest),
    ProtocolVersion = ProtocolVersion.V10a,
    TamperProtectionElements = new ITamperProtectionChannelBindingElement[] {
		new HmacSha1SigningBindingElement()
	}
};

Basically we are describing the oauth endpoints that will be used in the oauth dance, the oauth version, and the signing policy used in the service provider, which in this case is the method HMAC-SHA1 defined in RFC2104. Next we have to write our token manager, which must implement the interface IConsumerTokenManager (whic must implement the ITokenManager interface). Using the Alt-F10 Implement Interface of Visual Studio we get the skeleton of our class

public class ZeroLeggedTokenManager : IConsumerTokenManager
{
    public string ConsumerKey
    {
        get { throw new NotImplementedException(); }
    }

    public string ConsumerSecret
    {
        get { throw new NotImplementedException(); }
    }

    public void ExpireRequestTokenAndStoreNewAccessToken(string consumerKey, string requestToken, string accessToken, string accessTokenSecret)
    {
        throw new NotImplementedException();
    }

    public string GetTokenSecret(string token)
    {
        throw new NotImplementedException();
    }

    public TokenType GetTokenType(string token)
    {
        throw new NotImplementedException();
    }

    public void StoreNewRequestToken(DotNetOpenAuth.OAuth.Messages.UnauthorizedTokenRequest request, DotNetOpenAuth.OAuth.Messages.ITokenSecretContainingMessage response)
    {
        throw new NotImplementedException();
    }
}

Since we are using a 0-legged oauth, we have just to worry with the GetTokenSecret method, and the getter properties of our consumer key and consumer secret. Here is the final version of our token manager

public class ZeroLeggedTokenManager : IConsumerTokenManager
{
    private string _consumerKey;
    private string _consumerSecret;

    public ZeroLeggedTokenManager(string consumerKey, string consumerSecret)
    {
        _consumerKey = consumerKey;
        _consumerSecret = consumerSecret;
    }

    public string ConsumerKey { get { return _consumerKey; }}

    public string ConsumerSecret { get { return _consumerSecret; } }

    public void ExpireRequestTokenAndStoreNewAccessToken(string consumerKey, string requestToken, string accessToken, string accessTokenSecret)
    {
        throw new NotImplementedException();
    }

    public string GetTokenSecret(string token)
    {
        //In a 0-legged conversation only the consumer secret is used to sign the message
        return "";
    }

    public TokenType GetTokenType(string token)
    {
        throw new NotImplementedException();
    }

    public void StoreNewRequestToken(DotNetOpenAuth.OAuth.Messages.UnauthorizedTokenRequest request, DotNetOpenAuth.OAuth.Messages.ITokenSecretContainingMessage response)
    {
        throw new NotImplementedException();
    }
}

Now we can create our token manager

var tokenManager = new ZeroLeggedTokenManager(CONSUMER_KEY, CONSUMER_SECRET);

The CONSUMER_KEY and CONSUMER_SECRET are constants which contains the values of the consumer key and secret respectively.

Finally we can create our consumer and use it to make the call, using the method PrepareAuthorizeRequestAndSend

var zeroLeggedWebConsumer = new DotNetOpenAuth.OAuth.WebConsumer(providerDesc, tokenManager);

var response = zeroLeggedWebConsumer.PrepareAuthorizedRequestAndSend(
    new MessageReceivingEndpoint("https://bfcamara.uservoice.com/api/v1/users.json",
       HttpDeliveryMethods.GetRequest), "DUMMY");

When calling the PrepareAuthorizedRequestToken, we have to pass the endpoint to call, and the token to use, which will be used by the consumer to ask to our token manager what is the secret associated with this token. There is a little trick here: since our secret is always an empty secret returned by the token manager, it's quite irrelevant what you pass to the method, however it cannot be null or empty, which is not allowed. In our case we pass "DUMMY" as the token. (If you pass an empty or null token you will get an exception at runtime thrown by dotnetopenauth library).

Using Fiddler Web Debugger, you can check the content of the request/response

Now you can parse the response as you want.

Sumarizing, in this post, I demonstrated how to use the DotNetOpenAuth library to consume API's with a 0-legged OAuth.

OAuth: 2-legged vs 0-legged (UserVoice API as an example)

Tue, 23 Oct 2012 00:00:00 GMT

Nowadays it's very common to use OAuth as the method for clients to access server resources on behalf of a resource owner. OAuth also provides a process for end-users to authorize 3rd-party access to their server resources without sharing their credentials (tipically, a username and password pair), using user-agent redirections. Google, Twitter, Facebook, LinkedIn, and many other services providers, use OAuth in their APIs.

However, there is some debate in the community about the interpretation of the number of legs when using OAuth. The curious thing is that both RFCs RFC-5849 for OAuth 1.0 and RFC-6749 for OAuth 2.0, does not use the term leg at all. So, it seems the term leg is kind of an interpretation of the spec, and of course there are multiples interpretations in the community. There are those who consider that the number of legs are the number of steps involved to get the access token. And there are those who consider that the number of legs are the number of parties involved in OAuth dance. For me, it makes more sense to consider that the number of legs is the number of steps involved to obtain to the access token, which allow me to distinct between a 2-legged from a 0-legged oauth (more on that later).

The most used oauth dance uses a 3-leg process, and it seems that there is agreement about this 3-leg definition:

The consumer (the app that wants to access data) obtains a request token from the service provider
The consumer redirects the user to the service provider , using the request token obtained. The user logs in and grants the requested permissions on the service provider pages
The service provider redirects the user to the consumer site, and the consumer obtains the access token from the service provider

The division in the community is when refering to 2-legged OAuth (vs a 0-legged OAuth). In my point of view, the 0-legged OAuth consists of a consumer accessing the protected resources submitting OAuth signed requests using just the consumer key and the consumer secret. The process of obtaining these is defined by each service provider and is once per application, usually by the application developer. This is different from the 2-legged OAuth, which is a 3-legged OAuth without requiring the end-user to participate (missing step 2 and no redirections) . Andrew Arnott has a great post where he explains What is 2-legged OAuth. If you want to have a sense of the discussion about the 2-legged OAuth vs 0-legged OAuth, see the comments of his post.

I had to learn the distinction between the 2-legged OAuth and 0-legged OAuth when trying to integrate with UserVoice API. In the Getting Started section of Uservoice API documentation, it is said

There are three types of requests:

- Unauthenticated Requests- Requests you can make without any authentication other than passing your API Key. ex: Retrieving ideas or comments from a public forum.
- 3-legged OAuth requests- Requests made on the behalf of a user (see any API endpoints marked as requiring a User or Admin). ex: Voting on an idea, Creating a comment, Responding to a idea..
- 2-legged OAuth requests- Requests made on behalf of the client not on behalf of any specific user (see any API endpoints marked as requiring a Trusted Client) ex: Retrieving all users for a site, Getting a list of ideas for a private forum.

In my case I was developing the method to retrieve all users for a site, which is refered to be a 2-legged OAuth request. I am using the library dotnetopenauth, which it seems to me the most used oauth library in the .Net world.

In my point of view I had to get a request token first and then exchange it to get the access token, and finally use the access token when signing the requests to access resources. But when submitting the second request to get the access token I always got a 401 Unauthorized.

I decided to get some help from the support, and here is the answer that I received by email

Hi Bruno,

Our Developers figured out what's going on.

You're trying to do 2-legged OAuth basically like 3-legged OAuth without the user part. This is what most people mean by 2-legged OAuth. We don't support this type of 2-legged OAuth at this time.

What we call 2-legged OAuth in our docs is what some people have suggested calling 0-legged OAuth. It is basically signing requests with the client secret only.

In our examples we do this by creating an AccessToken object for a user. At least in the Ruby OAuth library, this sets the token and secret both to empty string. We then use this token to sign the requests. There is no need to ask the API for a request token or for an access token. You can just create a blank access token and start making requests.

Let me know if you have any more questions.

So what can we conclude from this? We have to understand from the service provider perspective what they are refering as a 2-legged oauth, since there are this 0-legged oauth which is different. In the next post I will show how to to this 0-legged oauth using dotnetopenauth, since there are some tricks to do it.

An inspirational story: The Mexican Fisherman

Tue, 16 Oct 2012 00:00:00 GMT

One of the books that I've read recently was the 4 hour work week from Timothy Ferriss. There are some ideas on the book that make me say that it's a worth reading, however I am skeptical about some of them. There is no doubt that the guy is a great marketer, which can be checked by the book title itself

The 4-Hour workweek - Escape the 9 - 5, live anywhere and join the new rich

When I first saw the book I thought "here we have another quick rich scheme scam" of the millions that exists on the internet. However the book was recommended by a friend of mine, and then I "took the shot". Since the average customer review on the Amazon is also very high, why not buy the book and read it. That's what I did. And as I said, it is a worth reading, and not a scam at all. The book title is exaggerated, but is a great title in a marketing point of view.

In this book I had contact with a great and inspirational story of the Mexican Fisherman. Here is the Story

An American businessman took a vacation to a small coastal Mexican village on doctor's orders. Unable to sleep after an urgent phone call from the office the first morning, he walked out to the pier to clear his head. A small boat with just one fisherman had docked, and inside the boat were several large yellowfin tuna. The American complimented the Mexican on the quality of his fish.

"How long did it take you to catch them?" the American asked.

"Only a little while," the Mexican replied in surprisingly good English.

"Why don't you stay out longer and catch more fish?" the American then asked.

"I have enough to support my family and give a few to friends," the Mexican said as he unloaded them into a basket.

"But... What do you do with the rest of your time?"

The Mexican looked up and smiled. "I sleep late, fish a little, play with my children, take a siesta with my wife, Julia, and stroll into the village each evening, where I sip wine and play guitar with my amigos. I have a full and busy life, senor."

The American laughed and stood tall. "Sir, I'm a Harvard M.B.A. and can help you. You should spend more time fishing, and with the proceeds, buy a bigger boat. In no time, you could buy several boats with the increased haul. Eventually, you would have a fleet of fishing boats."

He continued, "Instead of selling your catch to a middleman, you would sell directly to the consumers, eventually opening your own cannery. You would control the product, processing, and distribution. You would need to leave this small coastal fishing village, of course, and move to Mexico City, then to Los Angeles, and eventually New York City, where you could run your expanding enterprise with proper management."

The Mexican fisherman asked, "But, señor, how long will all this take?"

To which the American replied, "15-20 years. 25 tops."

"But what then, señor?"

The American laughed and said, "That's the best part. When the time is right, you would announce an IPO and sell your company stock to the public and become very rich. You would make millions."

"Millions, senor? Then what?"

"Then you would retire and move to a small coastal fishing village, where you would sleep late, fish a little, play with your kids, take a siesta with your wife, and stroll to the village in the evening where you could sip wine and play your guitar with your amigos.

I don't know who the author is, but this story had a profound impact on my feelings. Why? Because in the past I thought as the businessman, that I would need to be rich to have the freedom of how I would spend my time. In one of my previous posts I explained my big mistake as a founder which reflects this kind of thinking. Now I think like the Fisherman, and it's the road that I want to follow.

I'm back with a "fixed" heart

Mon, 08 Oct 2012 00:00:00 GMT

On June 21 I was the target of a cardiothoracic surgery. This surgery was to "fix" an aneurysm in the ascending aorta and to implant a mechanical aortic valve.

It all started 12 years ago, around the year 2000, when in a occupational medicine consultation, the physician detected a "murmur" in my heart . It was then that I began to be followed by a cardiologist, with periodic examinations on my heart. Basically I had a defect in the aortic valve and my cardiologist always told me that by the age of 50 I would have to replace the valve.

Throughout all these years, I have done several exams, and it was detected that the diameter of the ascending aorta was increasing. At the end of 2011 I was informed that that there was a risk of the artery to be ruptured, with potentially fatal consequences. Since the surgery would require to open the chest, then it makes sense take advantage and implant the valve, instead of waiting more years to open the chest again.

This period of sick leave, although quite painful, served to do some reflections and put some of the reading in day. This week I am returning to work and also in the hope to keep the blog more active.

Link post

Fri, 08 Jun 2012 00:00:00 GMT

http://www.bornontheweb.be/things-to-remember-when-setting-up-a-saas-bus

Link post

Wed, 23 May 2012 00:00:00 GMT

http://onstartups.com/tabid/3339/bid/84427/Freemium-Pricing-for-SaaS-Optimizing-Paid-Conversion-Upgrades.aspx

Link post

Thu, 17 May 2012 00:00:00 GMT

http://sivers.org/ss

Link post

Sun, 13 May 2012 00:00:00 GMT

http://24waystostart.com/2010/designer-architect-developer/

Designer, Architect, Developer: The three major skills that should be present in order to best build a successful web application

Link post

Thu, 10 May 2012 00:00:00 GMT

http://www.inc.com/jeff-haden/5-laws-of-business-youve-never-heard-of.html

Link post

Wed, 04 Apr 2012 00:00:00 GMT

http://dannydo.es/a-startup-despite-these-7-reasons/

Link post

Tue, 20 Mar 2012 00:00:00 GMT

http://steveblank.com/books-for-startups/

So you think you can dance?

Mon, 19 Mar 2012 00:00:00 GMT

Don't be fooled by the title, since I don’t have any dancing skills and probably never will. In fact my wife always said me that I am a lousy dancer. Her feet always suffer when we dance together.

The real question I want to ask is:

Do you think you can leave your day job and start your own company?

During my life, I’ve listen too many persons criticizing the company partners/directors, about some of the option behind made, or about some policies, etc.

It's legitimate, nobody is above critic. I also criticize some options. What makes me uncomfortable is listen sentences such as

"If I had my own company I..."

My question is immediately: “So, why don’t you?” or "So you think you can leave your day job and start your own company?"

Let me show you a chart

This chart shows my asset values (house mortgage not accounted for). The inflection point, at January of 2004, is when I left my job and started my own company. There were many fringe benefits that I’ve lost, such as car, fuel allowance, mobile phone, stock options, tuition, etc. Some of these perks, I had to buy/pay myself after leaving my job.

So you think you can handle this?

I think it will be interest to analyze the typical answers, “I have a mortgage to pay”, “I have kids and I need some security”, “I want to guarantee my retire plan”, but maybe in a future post.

P.S. I don’t intend to discourage anyone, but if you are doing the talk, you have at least be (seriously) willing to do the walk.

My first big mistake as founder

Mon, 06 Feb 2012 00:00:00 GMT

My first big mistake as a founder was my own ambition (which could be also seen as greed) and confusing the objective with the path to achieve it. But I'm getting ahead of the story.

When I, and my co-founders, founded Agilior, we did an exercise where each one had to describe the reasons (personal or professional), for starting our own company. In my case, there were basically two main reasons.

The professional reason was simple: professional achievement. I wanted to build a brand that would be recognized for his quality, integrity and honesty. I also wanted to build a place that was nice to work, with a culture of friendship, mutual help and wellness. Perhaps too naive, but that was what I thought at the time.

The personal reason was not so simple: getting my "freedom". In that exercise, I remember that I wrote the following sentence

"I want the freedom to leave at 3 pm so I can take my kids to the movies"

Freedom is a term that has many meanings. Josh Kaufman, from the Personal MBA project, wrote an excellent post on this feeling of wanting to build a business that supports your life. This freedom it's all I ambitioned and it was my personal reason to build my own company.

Of course this freedom presupposes a set of assumptions, which would imply an operational distance in the medium term. When I speak of operational distance, I mean not having to be involved full-time in the implementation of the projects we were developing. And to do that, we had to grow the company.

Some time after founding Agilior, I realized that much of this freedom that I wanted to achieve, I was already getting it, and that it wasn't necessary to grow the company. I started to managing my own time in my own terms, which allowed me have more free time for myself and my family, and able to work at "less common" hours. I never compromised my team, and I know when we work as a team, it is important to have the team members working at the same time. But the fact is that the freedom that I had ambitioned, I already had to a large extent.

The big mistake happened, when I became obsessed with growing the company to increase my operational distance, though, in retrospective, it was not necessary at all. When I and my co-founder were approached for the 1st time, by another company, to sell to them a part of the company, I saw it as an opportunity to make the company play in the 1st league (we were a small niche shop and profitable since day one), exponentially increase in sales, grow the company and have more freedom. And I can say it was me who insisted more on this step, thinking that everything would go well.(damn wishful thinking)

What happened was that the we sold 60% of the company (a few years later we would sell the remaining shares). What did not happen was the growth in sales (at least not in the promised terms), yet the company heacount has grown.

In the first transaction, we lost the financial control of the company since we now had only held 40% of the share, however we agreed that we would continue to have the operational control, allowing us to veto any decisions which we did not agree too. There was also a period of experimentation, where either party could roll back the transaction. During this period, no major changes occurred, except that we were “given” the opportunity to work on a new contract, a larger one compared to the ones that we were being used to. When the trial period ended, we decided to not rollback the transaction, and start working as a business partners.

In a retrospective the first transaction, we made the mistake of not writing on paper the expectations of each party.The trial period was not enough. And more important than to write the expectations, is to write what should happen if the expectations are not met. It is really really important.

What happened next, as I said earlier, was that sales have not grown . Moreover, most of the sales continued to be closed by my co-founder, and others by myself. The expected growth of sales was not met. But the other side started pressuring that we were not having the revenues as high as we should.

At some point, we realized that it was not worth continuing with this model, and we approached the other side to buy back the shares. The proposal was not accepted, and then we proceed with a proposal to sell the remaining of our shares. I am currently an employee of the company.

At end of the the day, I lost my freedom. And this was my biggest mistake. I was much happier with the company with three employees, with full autonomy in decision making, personal life, etc.. I was betrayed by my own greed.

A mistake not to be made again. If you ask me the reasons which lead me to make a company, I think I will answer it in the same way. Maybe I will place more emphasis on doing something what I really love. But the path will certainly be different. The most important thing is the path, and not the end. The path must give me the freedom that I want. There are times we have to do some trade-offs, I have a family with two children, and have to put the food on the table. The problem is when our greed goes far beyond our need and obscures the clarity of thought. Recently I read the book Anything You Want from Derek Sivers, the guy who created CD Baby. I loved the book, and I know understand Derek when he says

"I’m a minimalist. The less I own, the happier I am."

Welcome to my blog

Sat, 28 Jan 2012 00:00:00 GMT

One of my resolutions for 2012 was back to blog . I decided to kill my old blog (in fact, it was already dead), which is hosted in the Agilior's corporate blog, the company where I work and I was a co-founder (right now, I already haven't any part of the company's capital) and create a new blog with a personal touch,associated with the bfcamara.com domain.

The most of the content that I will publish is certainly related to my work, software development.

As one of the founders of Agilior, I hope also to share some of the stories that were part of my life as founder, and some of the mistakes I made along this route. There will be space also to personal subjects, however with lower frequency.

In one of my posts at the old blog, I explain that my life has been done of cycles. My perception is that a new cycle is about to come. I do not expect an easy time, and certainly I will have to leave the comfort zone. But "freedom" is one of the things that I value most, and to be honest, I like to be the leader and the captain of the ship", deciding where to go and when.

I hope I can maintain some activity in this blog.

Blog by Bruno Câmara

Exploring AWS WAF rules with OWASP Juice Shop

Web Application Firewall (WAF)

OWASP Top 10

OWASP Juice Shop

Installing Juice Shop Locally (Docker)

Installing Juice Shop in AWS AppRunner

Setting AWS WAF in Juice Shop App Runner

Testing WAF with SQL Injection

Testing WAF with Cross-Site Scripting (XSS)

Conclusion

Using AWS Cognito for Service-to-Service Authorization in ASP.NET Core

OAuth 2.0 - Client Credentials Grant flow 🔒

Amazon Cognito Setup ☁️

Creating a User Pool

Setting a Cognito domain

Creating a Resource Server for MyService

Create a Client for MyBFF

CloudFormation Template for Client Credentials Grant flow

MyService Setup (Resource Server)

MyBFF Setup (Client)

Conclusion

AWS Lambda - HTTP APIs and Concurrent Requests

HTTP APIs

Cold Starts

Minimizing Cold Starts

Can your HTTP API live with Cold Starts?

AWS Serverless alternatives to AWS Lambda

Conclusion

Visual Studio 2022 - Remote Testing

Migrating Azure Pipelines from Classic to YAML

🌲 CI/CD Pipelines as Code

🤼‍♂️ Unify Build and Release Pipelines

🏗️ Convert Task Groups into Templates

💻 Use Environments in YAML

💂 Use Approvals instead of Manual Trigger Deployments

📧 Stage Completed Notification

C# Static fields inline initialization - Be Lazy

✅ The root cause of the failing test

✂️ Explaining the issue with a simple scenario

🦹 Now the legacy code

👎 And now the failing test

👍 Let’s have full control and be lazy

Chasing a failed test in CI/CD pipeline

😨 You have broken the CI/CD pipeline

💻 It works on my machine

🕵️‍♂️ Investigation

VS 2019 on Windows - Debugging a unit test running in a Linux container

Preparing Interviews for a Software Engineer position

Be ready to write code in an interview

Practicing Code/SQL challenges

Big-O, Algorithms and Data Structures

Object Oriented and Design Patterns

Be ready to brainstorm about real problems

Be ready to be asked about past projects

Know about the interviewer as much as possible

Soft skills

Conclusion

DDD is fancy - Are we doing it right? Should we use it?

How are we doing DDD

Keep studying DDD

The Ubiquitous Language

Technical complexity

Don’t need to be “DDD All-In”

Conclusion

The Fear of Public Speaking

Excluding jobs from Succeeded Jobs in Hangfire Dashboard

Most Popular Programming Languages 1965-2019

From Tumblr to Gatsby

Blog Migration Requirements

Migration of Tumblr content to Markdown

Choose the design by using a Gatsby starter

Choose the platform to host the blog

Keep old URLs (no broken links)

Conclusion

Adopting Gatsby

Link post

Link post

Link post

Link post

Creating a Resource Server for `MyService`

Create a Client for `MyBFF`

`MyService` Setup (Resource Server)

`MyBFF` Setup (Client)