Recently I was working on securing ASP.NET Web API HTTP service that will be consumed by a large number of terminal devices installed securely in different physical locations, the main requirement was to authenticate calls originating from those terminal devices to the HTTP service and not worry about the users who are using it. So first thing came to my mind is to use one of the OAuth 2.0 flows which is Resource Owner Password Credentials Flow, but this flow doesn’t fit nicely in my case because the bearer access tokens issued should have expiry time as well they are non revocable by default, so issuing an access token with very long expiry time (i.e one year) is not the right way to do it.
After searching for couple of hours I found out that the right and maybe little bit complex way to implement this is to use HMAC Authentication (Hash-based Message Authentication Code).
The source code for this tutorial is available on GitHub.
What is HMAC Authentication?
It is a mechanism for calculating a message authentication code using a hash function in combination with a shared secret key between the two parties involved in sending and receiving the data (Front-end client and Back-end HTTP service) . The main use for HMAC to verify the integrity, authenticity, and the identity of the message sender.
So in simpler words the server provides the client with a public APP Id and shared secret key (API Key – shared only between server and client), this process happens only the first time when the client registers with the server.
After the client and server agrees on the API Key, the client creates a unique HMAC (hash) representing the request originated from it to the server. It does this by combining the request data and usually it will contain (Public APP Id, request URI, request content, HTTP method, time stamp, and nonce) in order to produce a unique hash by using the API Key. Then the client sends that hash to the server, along with all information it was going to send already in the request.
Once the server revives the request along with the hash from the client, it tries to reconstruct the hash by using the received request data from the client along with the API Key, once the hash is generated on the server, the server will be responsible to compare the hash sent by the client with the regenerated one, if they match then the server consider this request authentic and process it.
Flow of using API Key – HMAC Authentication:
Note: First of all the server should provide the client with a public (APP Id) and shared private secret (API Key), the client responsibility is to store the API Key securely and never share it with other parties.
Flow on the client side:
- Client should build a string by combining all the data that will be sent, this string contains the following parameters (APP Id, HTTP method, request URI, request time stamp, nonce, and Base 64 string representation of the request pay load).
- Note: Request time stamp is calculated using UNIX time (number of seconds since Jan. 1st 1970) to overcome any issues related to a different timezone between client and server. Nonce: is an arbitrary number/string used only once. More about this later.
- Client will hash this large string built in the first step using a hash algorithm such as (SHA256) and the API Key assigned to it, the result for this hash is a unique signature for this request.
- The signature will be sent in the Authorization header using a custom scheme such as”amx”. The data in the Authorization header will contain the APP Id, request time stamp, and nonce separated by colon ‘:’. The format for the Authorization header will be like: [Authorization: amx APPId:Signature:Nonce:Timestamp].
- Client send the request as usual along with the data generated in step 3 in the Authorization header.
Flow on the server side:
- Server receives all the data included in the request along with the Authorization header.
- Server extracts the values (APP Id, Signature, Nonce and Request Time stamp) from the Authorization header.
- Servers looks for the APP Id in a certain secure repository (DB, Configuration file, etc…) to get the API Key for this client.
- Assuming the server was able to look up this APP Id from the repository, it will be responsible to validate if this request is a replay request and reject it, so it will prevent the API from any replay attacks. This is why we’ve used a request time stamp along with nonce generated at the client, and both values have been included into HMAC signature generation. The server will depend on the nonce to check if it was used before within certain acceptable bounds, i.e. 5 minutes. More about this later.
- Server will rebuild a string containing the same data received in the request by adhering to the same parameters orders and encoding followed in the client application, usually this agreement is done up front between the client application and the back-end service and shared using proper documentation.
- Server will hash the string generated in previous step using the same hashing algorithm used by the client (SHA256) and the same API Key obtained from the secure repository for this client.
- The result of this hash function (signature) generated at the server will be compared to the signature sent by the client, if they are equal then server will consider this call authentic and process the request, otherwise the server will reject the request and returns HTTP status code 401 unauthorized.
Important note:
- Client and server should generate the hash (signature) using the same hashing algorithm as well adhere to the same parameters order, any slight change including case sensitivity when implementing the hashing will result in totally different signature and all requests from the client to the server will get rejected. So be consistent and agree on how to generate the signature up front and in clear way.
- This mechanism of authentication can work without TLS (HTTPS), as long as the client is not transferring any confidential data or transmitting the API Key. It is recommended to consume it over TLS. But if you can’t use TLS for any other reason you will be fine if you transmit data over HTTP.
Sounds complicated? Right? Let’s jump to the implementation to make this clear.
I’ll start by showing how to generate APP Id and strong 265 bits key which will act as our API Key, this usually will be done on the server and provided to the client using a secure mechanism (Secure admin server portal). There is nice post here explains why generating APP Ids and API Keys is more secure than issuing username and password.
Then I’ll build a simple console application which will act as the client application, lastly I’ll build HTTP service using ASP.NET Web API and protected using HMAC Authentication using the right filter “IAuthenticationFilter”, so let’s get started!
The source code for this tutorial is available on GitHub.
Section 1: Generating the Shared Private Key (API Key) and APP Id
As I stated before this should be done on the server and provided to the client prior the actual use, we’ll use symmetric key cryptographic algorithm to issue 256 bit key, the code will be as the below:
|
1 2 3 4 5 6 |
using (var cryptoProvider = new RNGCryptoServiceProvider()) { byte[] secretKeyByteArray = new byte[32]; //256 bit cryptoProvider.GetBytes(secretKeyByteArray); var APIKey = Convert.ToBase64String(secretKeyByteArray); } |
And for the APP Id you can generate a GUID, so for this tutorial let’s assume that our APPId is: 4d53bce03ec34c0a911182d4c228ee6c and our APIKey generated is: A93reRTUJHsCuQSHR+L3GxqOJyDmQpCgps102ciuabc= and assume that our client application has received those 2 pieces of information using a secure channel.
Section 2: Building the Client Application
Step 1: Install Nuget Package
Add new empty solution named “WebApiHMACAuthentication” then add new console application named “HMACAuthentication.Client”, then install the below HTTPClient Nuget package which help us to issue HTTP requests.
|
1 |
Install-Package Microsoft.AspNet.WebApi.Client -Version 5.2.2 |
Step 2: Add POCO Model
We’ll issue HTTP POST request in order to demonstrate how we can include the request body in the signature, so we’ll add simple model named “Order”, add new class named “Order” and paste the code below:
|
1 2 3 4 5 6 7 |
public class Order { public int OrderID { get; set; } public string CustomerName { get; set; } public string ShipperCity { get; set; } public Boolean IsShipped { get; set; } } |
Step 3: Call the back-end API using HTTPClient
Now we’ll use the HTTPClient library installed earlier to issue HTTP POST request to the API we’ll build in the next section, so open file “Program.cs” and paste the code below:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
static void Main(string[] args) { RunAsync().Wait(); } static async Task RunAsync() { Console.WriteLine("Calling the back-end API"); string apiBaseAddress = "http://localhost:43326/"; CustomDelegatingHandler customDelegatingHandler = new CustomDelegatingHandler(); HttpClient client = HttpClientFactory.Create(customDelegatingHandler); var order = new Order { OrderID = 10248, CustomerName = "Taiseer Joudeh", ShipperCity = "Amman", IsShipped = true }; HttpResponseMessage response = await client.PostAsJsonAsync(apiBaseAddress + "api/orders", order); if (response.IsSuccessStatusCode) { string responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); Console.WriteLine("HTTP Status: {0}, Reason {1}. Press ENTER to exit", response.StatusCode, response.ReasonPhrase); } else { Console.WriteLine("Failed to call the API. HTTP Status: {0}, Reason {1}", response.StatusCode, response.ReasonPhrase); } Console.ReadLine(); } |
What we implemented here is basic, we just issuing HTTP POST to the end point “/api/orders” including serialized order object, this end point is protected using HMAC Authentication (More about this later in post), and if the response status returned is 200 OK, then we are printing the response returned.
What worth nothing here that I’m using a custom delegation handler named “CustomDelegatingHandler”. This handler will help us to intercept the request before sending it so we can do the signing process and creating the signature there.
Step 4: Implement the HTTPClient Custom Handler
HTTPClient allows us to create custom message handler which get created and added to the request message handlers chain, the nice thing here that this handler will allow us to write out custom logic (logic needed to build the hash and set in the Authorization header before firing the request to the back-end API), so in the same file “Program.cs” add new class named “CustomDelegatingHandler” and paste the code below:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
public class CustomDelegatingHandler : DelegatingHandler { //Obtained from the server earlier, APIKey MUST be stored securely and in App.Config private string APPId = "4d53bce03ec34c0a911182d4c228ee6c"; private string APIKey = "A93reRTUJHsCuQSHR+L3GxqOJyDmQpCgps102ciuabc="; protected async override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) { HttpResponseMessage response = null; string requestContentBase64String = string.Empty; string requestUri = System.Web.HttpUtility.UrlEncode(request.RequestUri.AbsoluteUri.ToLower()); string requestHttpMethod = request.Method.Method; //Calculate UNIX time DateTime epochStart = new DateTime(1970, 01, 01, 0, 0, 0, 0, DateTimeKind.Utc); TimeSpan timeSpan = DateTime.UtcNow - epochStart; string requestTimeStamp = Convert.ToUInt64(timeSpan.TotalSeconds).ToString(); //create random nonce for each request string nonce = Guid.NewGuid().ToString("N"); //Checking if the request contains body, usually will be null wiht HTTP GET and DELETE if (request.Content != null) { byte[] content = await request.Content.ReadAsByteArrayAsync(); MD5 md5 = MD5.Create(); //Hashing the request body, any change in request body will result in different hash, we'll incure message integrity byte[] requestContentHash = md5.ComputeHash(content); requestContentBase64String = Convert.ToBase64String(requestContentHash); } //Creating the raw signature string string signatureRawData = String.Format("{0}{1}{2}{3}{4}{5}", APPId, requestHttpMethod, requestUri, requestTimeStamp, nonce, requestContentBase64String); var secretKeyByteArray = Convert.FromBase64String(APIKey); byte[] signature = Encoding.UTF8.GetBytes(signatureRawData); using (HMACSHA256 hmac = new HMACSHA256(secretKeyByteArray)) { byte[] signatureBytes = hmac.ComputeHash(signature); string requestSignatureBase64String = Convert.ToBase64String(signatureBytes); //Setting the values in the Authorization header using custom scheme (amx) request.Headers.Authorization = new AuthenticationHeaderValue("amx", string.Format("{0}:{1}:{2}:{3}", APPId, requestSignatureBase64String, nonce, requestTimeStamp)); } response = await base.SendAsync(request, cancellationToken); return response; } } |
What we’ve implemented above is the following:
- We’ve hard coded the APP Id and API Key values obtained earlier from the server, usually you need to store those values securely in app.config.
- We’ve got the full request URI and safely Url Encoded it, so in case there is query strings sent with the request they will safely encoded, as well we’ve read the HTTP method used, in our case it will be POST.
- We’ve calculated the time stamp for the request using UNIX timing (number of seconds since Jan. 1st 1970). This will help us to avoid any issues might happen if the client and the server resides in two different time zones.
- We’ve generated a random nonce for this request, the client should adhere to this and should send a random string per method call.
- We’ve checked if the request contains a body (it will contain a body if the request of type HTTP POST or PUT), if it contains a body; then we will md5 hash the body content then Base64 the array, we are doing this to insure the authenticity of the request and to make sure no one tampered with the request during the transmission (in case of transmitting it over HTTP).
- We’ve built the signature raw data by concatenating the parameters (APPId, requestHttpMethod, requestUri, requestTimeStamp, nonce, requestContentBase64String) without any delimiters, this data will get hashed using HMACSHA256 algorithm.
- Lastly we’ve applied the hashing algorithm using the API Key then base64 the result and combined the (APPId:requestSignatureBase64String:nonce:requestTimeStamp) using ‘:’ colon delimiter and set this combined string in the Authorization header for the request using a custom scheme named “amx”. Notice that the nonce and time stamp are included in creating the request signature as well they are sent as plain text values so they can be validated on the server to protect our API from replay attacks.
We are done of the client part, now let’s move to building the Web API which will be protected using HMAC Authentication.
Section 3: Building the back-end API
Step 1: Add the Web API Project
Add new Web application project named “HMACAuthentication.WebApi” to our existing solution “WebApiHMACAuthentication”, the template for the API will be as the image below (Web API core dependency checked) or you can use OWIN as we did in previous tutorials:
Step 2: Add Orders Controller
We’ll add simple controller named “Orders” controller with 2 simple HTTP methods, as well we’ll add the same model “Order” we already added in the client application, so add new class named “OrdersController” and paste the code below, nothing special here, just basic Web API controller which is not protected and allows anonymous calls (we’ll protect it later in the post).
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
[RoutePrefix("api/Orders")] public class OrdersController : ApiController { [Route("")] public IHttpActionResult Get() { ClaimsPrincipal principal = Request.GetRequestContext().Principal as ClaimsPrincipal; var Name = ClaimsPrincipal.Current.Identity.Name; return Ok(Order.CreateOrders()); } [Route("")] public IHttpActionResult Post(Order order) { return Ok(order); } } #region Helpers public class Order { public int OrderID { get; set; } public string CustomerName { get; set; } public string ShipperCity { get; set; } public Boolean IsShipped { get; set; } public static List<Order> CreateOrders() { List<Order> OrderList = new List<Order> { new Order {OrderID = 10248, CustomerName = "Taiseer Joudeh", ShipperCity = "Amman", IsShipped = true }, new Order {OrderID = 10249, CustomerName = "Ahmad Hasan", ShipperCity = "Dubai", IsShipped = false}, new Order {OrderID = 10250,CustomerName = "Tamer Yaser", ShipperCity = "Jeddah", IsShipped = false }, new Order {OrderID = 10251,CustomerName = "Lina Majed", ShipperCity = "Abu Dhabi", IsShipped = false}, new Order {OrderID = 10252,CustomerName = "Yasmeen Rami", ShipperCity = "Kuwait", IsShipped = true} }; return OrderList; } } #endregion |
Step 3: Build the HMAC Authentication Filter
We’ll add all our logic responsible for re-generating the signature on the Web API and comparing it with signature received by the client in an Authentication Filter. The authentication filter is available in Web API 2 and it should be used for any authentication purposes, in our case we will use this filter to write our custom logic which validates the authenticity of the signature received by the client. The nice thing about this filter that it run before any other filters especially the authorization filter, I’ll borrow the image below from a great article about ASP.NET Web API Security Filters by Badrinarayanan Lakshmiraghavan to give you better understanding on where the authentication filter resides.
.png)
Now add new folder named “Filters” then add new class named “HMACAuthenticationAttribute” which inherits from “Attribute” and implements interface “IAuthenticationFilter”, then paste the code below:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 |
public class HMACAuthenticationAttribute : Attribute, IAuthenticationFilter { private static Dictionary<string, string> allowedApps = new Dictionary<string, string>(); private readonly UInt64 requestMaxAgeInSeconds = 300; //5 mins private readonly string authenticationScheme = "amx"; public HMACAuthenticationAttribute() { if (allowedApps.Count == 0) { allowedApps.Add("4d53bce03ec34c0a911182d4c228ee6c", "A93reRTUJHsCuQSHR+L3GxqOJyDmQpCgps102ciuabc="); } } public Task AuthenticateAsync(HttpAuthenticationContext context, CancellationToken cancellationToken) { var req = context.Request; if (req.Headers.Authorization != null && authenticationScheme.Equals(req.Headers.Authorization.Scheme, StringComparison.OrdinalIgnoreCase)) { var rawAuthzHeader = req.Headers.Authorization.Parameter; var autherizationHeaderArray = GetAutherizationHeaderValues(rawAuthzHeader); if (autherizationHeaderArray != null) { var APPId = autherizationHeaderArray[0]; var incomingBase64Signature = autherizationHeaderArray[1]; var nonce = autherizationHeaderArray[2]; var requestTimeStamp = autherizationHeaderArray[3]; var isValid = isValidRequest(req, APPId, incomingBase64Signature, nonce, requestTimeStamp); if (isValid.Result) { var currentPrincipal = new GenericPrincipal(new GenericIdentity(APPId), null); context.Principal = currentPrincipal; } else { context.ErrorResult = new UnauthorizedResult(new AuthenticationHeaderValue[0], context.Request); } } else { context.ErrorResult = new UnauthorizedResult(new AuthenticationHeaderValue[0], context.Request); } } else { context.ErrorResult = new UnauthorizedResult(new AuthenticationHeaderValue[0], context.Request); } return Task.FromResult(0); } public Task ChallengeAsync(HttpAuthenticationChallengeContext context, CancellationToken cancellationToken) { context.Result = new ResultWithChallenge(context.Result); return Task.FromResult(0); } public bool AllowMultiple { get { return false; } } private string[] GetAutherizationHeaderValues(string rawAuthzHeader) { var credArray = rawAuthzHeader.Split(':'); if (credArray.Length == 4) { return credArray; } else { return null; } } } |
Basically what we’ve implemented is the following:
- The class “HMACAuthenticationAttribute” derives from “Attribute” class so we can use it as filter attribute over our controllers or HTTP action methods.
- The constructor for the class currently fill a dictionary named “allowedApps”, this is for the demo only, usually you will store the APP Id and API Key in a database along with other information about this client.
- The method “AuthenticateAsync” is used to implement the core authentication logic of validating the incoming signature in the request
- We make sure that the Authorization header is not empty and it contains scheme of type “amx”, then we read the Authorization header value and split its content based on the delimiter we’ve specified earlier in client ‘:’.
- Lastly we are calling method “isValidRequest” where all the magic of reconstructing the signature and comparing it with the incoming signature happens. More about implementing this in step 5.
- Incase the Authorization header is incorrect or the result of executing method “isValidRequest” returns false, we’ll consider the incoming request as unauthorized and we should return an authentication challenge to the response, this should be implemented in method “ChallengeAsync”, to do so lets implement the next step.
Step 4: Add authentication challenge to the response
To add authentication challenge to the unauthorized response copy and paste the code below in the same file “HMACAuthenticationAttribute.cs”, basically we’ll add “WWW-Authenticate” header to the response using our “amx” custom scheme . You can read more about the details of this implementation here.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
public class ResultWithChallenge : IHttpActionResult { private readonly string authenticationScheme = "amx"; private readonly IHttpActionResult next; public ResultWithChallenge(IHttpActionResult next) { this.next = next; } public async Task<HttpResponseMessage> ExecuteAsync(CancellationToken cancellationToken) { var response = await next.ExecuteAsync(cancellationToken); if (response.StatusCode == HttpStatusCode.Unauthorized) { response.Headers.WwwAuthenticate.Add(new AuthenticationHeaderValue(authenticationScheme)); } return response; } } |
Step 5: Implement the method “isValidRequest”.
The core implementation of reconstructing the request parameters and generating the signature on the server happens here, so let’s add the code then I’ll describe what this method is responsible for, open file “HMACAuthenticationAttribute.cs” again and paste the code below in class “HMACAuthenticationAttribute”:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 |
private async Task<bool> isValidRequest(HttpRequestMessage req, string APPId, string incomingBase64Signature, string nonce, string requestTimeStamp) { string requestContentBase64String = ""; string requestUri = HttpUtility.UrlEncode(req.RequestUri.AbsoluteUri.ToLower()); string requestHttpMethod = req.Method.Method; if (!allowedApps.ContainsKey(APPId)) { return false; } var sharedKey = allowedApps[APPId]; if (isReplayRequest(nonce, requestTimeStamp)) { return false; } byte[] hash = await ComputeHash(req.Content); if (hash != null) { requestContentBase64String = Convert.ToBase64String(hash); } string data = String.Format("{0}{1}{2}{3}{4}{5}", APPId, requestHttpMethod, requestUri, requestTimeStamp, nonce, requestContentBase64String); var secretKeyBytes = Convert.FromBase64String(sharedKey); byte[] signature = Encoding.UTF8.GetBytes(data); using (HMACSHA256 hmac = new HMACSHA256(secretKeyBytes)) { byte[] signatureBytes = hmac.ComputeHash(signature); return (incomingBase64Signature.Equals(Convert.ToBase64String(signatureBytes), StringComparison.Ordinal)); } } private bool isReplayRequest(string nonce, string requestTimeStamp) { if (System.Runtime.Caching.MemoryCache.Default.Contains(nonce)) { return true; } DateTime epochStart = new DateTime(1970, 01, 01, 0, 0, 0, 0, DateTimeKind.Utc); TimeSpan currentTs = DateTime.UtcNow - epochStart; var serverTotalSeconds = Convert.ToUInt64(currentTs.TotalSeconds); var requestTotalSeconds = Convert.ToUInt64(requestTimeStamp); if ((serverTotalSeconds - requestTotalSeconds) > requestMaxAgeInSeconds) { return true; } System.Runtime.Caching.MemoryCache.Default.Add(nonce, requestTimeStamp, DateTimeOffset.UtcNow.AddSeconds(requestMaxAgeInSeconds)); return false; } private static async Task<byte[]> ComputeHash(HttpContent httpContent) { using (MD5 md5 = MD5.Create()) { byte[] hash = null; var content = await httpContent.ReadAsByteArrayAsync(); if (content.Length != 0) { hash = md5.ComputeHash(content); } return hash; } } |
What we’ve implemented here is the below:
- We’ve validated that public APPId received is registered in our system, if it is not we’ll return false and will return unauthorized response.
- We’ve checked if the request received is a replay request, this means that checking if the nonce received by the client is used before, currently I’m storing all the nonce received by the client in Cache Memory for 5 minutes only, so for example if the client generated a nonce “abc1234” and send it with a request, the server will check if this nonce is used before, if not it will store the nonce for 5 minutes, so any request coming with same nonce during the 5 minutes window will consider a replay attack, if the same nonce “abc1234” is used after 5 minutes then this is fine and the request is not considered a replay attack.
- But there might be an evil person that might try to re-post the same request using the same nonce after the 5 minutes window, so the request time stamp becomes handy here, the implementation is comparing the current server UNIX time with the request UNIX time from the client, if the request age is older than 5 minutes too then it is rejected and the the evil person has no possibility to fake the request time stamp and send fresher one because we’ve already included the request time stamp in the signature raw data, so any change on it will result into new signature and it will not match the client incoming signature.
- Note: If your API is published on different nodes on web farm, then you can store those nonce using Microsoft Azure Cache or Redis server, do not store them in DB because you need fast rad access.
- Last step is we’ve implemented is to md5 hash the request body content if it is available (POST, PUT methods), then we’ve built the signature raw data by concatenating the parameters (APPId, requestHttpMethod, requestUri, requestTimeStamp, nonce, requestContentBase64String) without any delimiters. It is a MUST that both parties use the same data format to produce the same signature, the data eventually will get hashed using the same hashing algorithm and API Key used by the client. If the incoming client signature equals the signature generated on the server then we’ll consider this request authentic and will process it.
Step 5: Secure the API End Points:
Final thing to do here is to attribute the protected end points or controllers with this new authentication filter attribute, so open controller “Orders” and add the attribute “HMACAuthentication” as the code below:
|
1 2 3 4 5 6 |
[HMACAuthentication] [RoutePrefix("api/Orders")] public class OrdersController : ApiController { //Controller implementation goes here } |
Conclusion:
- In my opinion HMAC authentication is more complicated than OAuth 2.0 but in some situations you need to use it especially if you can’t use TLS, or when you are building HTTP service that will be consumed by terminals or devices which storing the API Key in it is fine.
- I’ve read that OAuth 1.0a is very similar to this approach, I’m not an expert of this protocol and I’m not trying to reinvent the wheel, I want to build this without the use of any external library, so for anyone reading this and have experience with OAuth 1.0a please drop me a comment telling the differences/ similarities about this approach and OAuth 1.0a.
That’s all for now folks! Please drop me a comment if you have better way implementing this or you spotted something that could be done in a better way.
The source code for this tutorial is available on GitHub.
Follow me on Twitter @tjoudeh
References:
- Designing a Secure REST (Web) API without OAuth by Riyad Kalla
- Security Framework by Eran Hammer.
- What is HMAC Authentication and why is it useful? by Mark Wolfe.
- HMAC authentication in ASP.NET Web API by Piotr Walat.
- Image credit: https://flic.kr/p/6UKyJq
Good article. To solve the expiry key problem, we wrote our own authentication API Keys to be effectively GUID’s held in our TriSys CRM system. This allows us to expire or limit some access to non-paying customers in real-time without having to re-issue keys. We can then turn on/off any feature of our product, even those not invented yet, by using our own secure/hidden internal SQL Server database. Of course we cannot share this with you, so congratulations on supplying a solution this way.
Thanks Garry for your comment, indeed the key expiry rotation and expiry is always an issue with HMAC authentication, your custom solution sounds interesting.
How do you handle the time on the server and client if it differ more than five minutes, I do not mean time zones, but if the clock is incorrectly set on the client?
In some cases the server can be built to distinguish bad time stamps (clock skew) and respond to the client by returning it is current UNIX time. Then the client is responsible to adjust the his clock and resend the request with correct TS, otherwise they will never be able to communicate.
Wow – great post! The level of detail is really good. This could also be applied to nodejs server to IIS communication as long as the encryption methods matched. And thanks for providing reference articles for further research. Good stuff. So good that I just subscribed to the updates 🙂
Thanks for taking the time to read the post and comment, glad that you find it detailed and useful. Thanks for subscribing too!
Good article! But I would recommend to switch the signature calculation algorithm to the one that used by the Amazon AWS: http://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html (you may simplify it a little bit though, remove “scope” for example, etc). It is well tested.
For example, this calculation is not safe in general:
string signatureRawData = String.Format(“{0}{1}{2}{3}{4}{5}”, APPId, requestHttpMethod, requestUri, requestTimeStamp, nonce, requestContentBase64String);
because it is theoretically possible to authenticate modified queries (via MITMA) by moving part of the data from one variable to another (like using the start of the nonce as the timestamp end, etc).
Also, be careful with using static System.Runtime.Caching.MemoryCache in IIS, because when application pool gets recycled it will be possible to send query with the same nonce.
Thanks Funbit for your comment, regarding the second point you are correct, if IIS pool is recycled all the saved nonce will be flushed, that why I mention used Redis service or Azure Cache is better to store those nonce.
But for the first point I see this is hard to happen, the hacker who is trying to sniff the request will never be able to regenerate new request without the Api Key, so how he will be able to move parts of the data and regenerate new signature without the Api key? Please let me know if I’m missing something here.
The point is that the hacker doesn’t need the Api Key to modify the request. For example, lets say we used the following information on the client side to sign the request:
nonce: 1234567890
timestamp: 9876543210
so the Auth header will become:
{appId}:{requestSignature}:1234567890:9876543210
Then, the hacker gets the request and modifies the Auth header this way (moves zero):
{appId}:{requestSignature}:123456789:09876543210
The signature of the the modified request will be exactly the same as the original one, because this format string: “{0}{1}{2}{3}{4}{5}” just joins the strings, but nonce and timestamp will be different.
Of course this is just a simple example, allowing to perform simple replay attack, but this is just the first one came to my mind… there might be other and more dangerous combinations.
Sorry, just noticed that in your format string timestamp goes before the nonce, but I think you get the idea 🙂
Hi again,
Well it seems you did not read the post till the and see how I’m reconstructing the signature at the server side, the plain nonce and time stamp sent in the authorization header to the server are used again in rebuilding the signature at the server side, the signature includes both plain text values (nonce and time stamp).
So in your example if the hacker changed any of both by moving a digit, and when the request is received at the server it will reconstruct the signature again using the new modified values and this will result in totally new signature and the request is rejected.
You scenario is valid if I’m not including the nonce and time stamp in building the request signature.
Thank you.
Hi,
I’m sorry, it seems that my explanation wasn’t clear enough so you didn’t understand the vulnerability. Of course, the signature is calculated on the server side again to check the request, but it does not matter if you give a hacker an ability to play with the values included in the signature calculation. I have successfully reproduced the attack on my side. The first one is done by moving part of the timestamp into the nonce (here your format string is lucky, because you put timestamp value before the nonce, so you cannot replay just by moving zeroes to the begging of the timestamp which is usually very useful from the hacker’s point of view, so to perform the attack I had to turn off timestamp validation on the server side). The second one is done by moving part of the nonce into the request URI. I have uploaded Fiddler’s log here:
http://pastebin.com/63mpyVua
so you can see for yourself. Please note that the HMAC digests are equal and valid (1st=2nd(malformed), and 3rd=4th(malformed)), so the hacker does not need to know the original App Key.
To make the authorization safer you should use AWS like approach when you strictly define each part of the HMAC string and use “\n” separator to split the parts. Also, it is better to include all the values in the HTTP headers so the Authorization HTTP header contains only HMAC digest and App Id.
Best regards,
Funbit
Thanks again for your comment, it will be great if you forked the repo and issued pull request.
UPD: made a mistake: in the second attack I moved part of the *timestamp* to the URI (not from the nonce).
I’m sorry, unfortunately I don’t have spare time now 🙁
But I have uploaded my implementation to the pastebin, so you may take some parts from there if you want: http://pastebin.com/TibwZnXG (SignRequest is used to sign a given HttpClient request, IsSignatureValid is used to check a request on the WebApi side).
Happy Christmas! 🙂
Regards,
Funbit
That was a really informative post.
I’m looking at doing authentication for a mobile app, where a user logs in with an email and password. I’m wondering if a modification of your method would be ok.
Instead of an API key, would it be ok (safe) to use the user’s email + hashed password. Those 2 combined and hashed would be the “API Key”. Then you would send the email in the body.
I would get the benefits of the time stamp.
Note: My app will work over HTTPS, so the email address would not be snooped.
Do you think that Is overkill / useful / useless ?
Regards
Anthony
Hi Anthony,
This approach should not be used if you are building an API to be consumed from mobile applications, you should not embed/store the API Key/user name or password in your mobile application, the right way to do this is to use OAuth 2.0 resource owner credentials flow (bearer token), the good thing that I’ve written detailed blog post about it along with demo application and source code, so check it here.
Hi Taiseer,
Great post. Learned a lot of things.
I am developing a ASP.NET Web API for use in mobile applications (Windows Phone). This is a company app and authorization is handled via Web API based on parameters in POST form body. Why do you not recommend HMAC for this? API ID/Key can be securely embedded as constants in the app. Why use OAuth?
Also, could I just use HTTPS or a key based authentication is better?
Regards,
Sid
Well OAuth 2.0 is intended to secure HTTP services for broad range of clients (Mobile apps, Desktop, JS apps, etc..) the HMAC authentication is very close to OAuth 1.0 specifications. While there is complexity in signing the request on both sides and validating the the hash values. My recommendation is to use OAuth 2.0 for securing HTTP services and use the HMAC auth in scnarios where you can’t use the OAuth 2.0 flows.
Please note that HTTPS is mandatory when using OAuth 2.0.
Thank you for the reply.
Currently the service is in testing mode. So, for now I have opted HMAC.
Planning on getting HTTPS. Then I might switch to OAuth2.0.
Although, from what i’ve read, with HTTPS, my requests will automatically be secure/encrypted. Should I go for OAuth2.0 in that case?
Once you send bearer tokens then you must to do this with HTTPS, so you protect your API from MIT attacks and no one steal your bearer tokens.
You re-invented OAuth 1.0 Taiseer! It sounds like it fit your use case very well. One difference is you sign the entire request body (obviously, since it’s an application/json request, whereas OAuth 1.0 doesn’t know what the hell to do with non application/x-www-form-urlencoded requests).
Hi Roatin, thanks for your comment.
Well I guess it is so close to OAuth 1.0, but I think there is little difference here, and btw Eran Hammer (one of the advocate authors for OAuth 1.0) built an HTTP authentication scheme using a message authentication code (MAC) called HAWK, I knew about this after I wrote this blog post, the HAWK is very close to what I did in this post 🙂
Nice post Taiseer! Even though it may be close to OAuth 1.0, its good to see someone approach it like you did and explain the process.
Cheers,
Adrian.
Thank you, you are right it is very close but knowing how this thing happen without using black box libraries is good thing, also check the HWAK authentication, very close to what I’ve implemented.
Great post
Great post! You call the hash message authentication code a signature in your code. This is something confusing because it isn’t a signature strictly speaking.
Furthermore I prefer the approach to use a timestamp in milliseconds per client. This timestamp will be stored in memory in a static per application Id. No extra nonce is necessary. As long as the list exists no one can reply in any way. Furthermore by storing it “static”, there is no need to use IIS cache. However, this list is also lost when the app pool is recycled.
Furthermore, it is not clear to me why the values where the HMAC is calculated from contain a hash too? The HMAC itself is a secure hash so why add another one?
Like funbit says, the values shouldn’t be simply concatenated but merged using value-size,value-size combinations: e.g.:
“myappid:7|post:4|….”
Ignore my latest comment. It is now clear to me it is the content hash.
Good article, It explains the inner workings of filters in combination with security a lot.
Just another thing. Timing attacks are not covered here. You can try to form the correct HMAC by running a timing attack. To prevent that, the comparison: “if incomingBase64Signature.Equals(…” should be replaced with a time independent comparison function.
Can you elaborate more please? The time stamp is included in the request string which will be used to generate the signature, so how timing attack can happen?
Well, a timing attack, is a way to find out the HMAC (in this case) by measuring the time it takes to compare the 2 HMACs:
– The HMAC send with the request;
– and the HMAC calculated within the service.
It is theoretically possible to find out how many characters of both HMAC matches by measuring the time it takes to handle the request. The time it takes depends on the time it takes to compare 2 strings, which you do in the incomingBase64Signature.Equals() function. The Equals function is not time independent, which means that if a 2 strings matches it takes less time to compare as where 2 strings not matches at all and all there is in between.
Again, it is theoretically possible to guess the HMAC. It is questionable if you want to implemented it here. It depends on the level of security you need. Maybe it is good to mention it (what I just did 😉
Hmm to be honest this an advance scenario that I didn’t think of 🙂 Thanks again for mentioning it 🙂
Hi Taiseer, thank you for the thorough write-up. I built a Web API 2 app and a client app, applied the API Key – HMAC Authentication as described, and they worked like a charm from end to end. However, when I deploy the Web API behind a reverse proxy server, I get a “401 Unauthorized” when I attempt to call it from the client app. If I strip the API Key – HMAC Authentication code from both apps, I can call it fine (but obviously it’s no longer secured…). Does this have anything to do with a missing Proxy-Authenticate Header? I found this article too, but didn’t have any success in applying the suggested solutions: http://stackoverflow.com/questions/299940/how-should-i-set-the-default-proxy-to-use-default-credentials Any other ideas or alternative solutions?
Hi Anthony,
I believe there will be difference in the request body between your client App and the request body your receive at the sever, you need to inspect those carefully and make sure they are 100% identical, If there is issue with the proxy then you will receive 407 status code not 401. As well check this implementation for similar approach called HAWK authentication, this one is mature and written by Eran Hammer.
I should add that it worked like a charm from end to end locally (localhost) and from within our intranet, but when I try to call it through the public facing reverse web proxy (https), I get 401. That’s with the HMAC code in place. If I strip the HMAC, I get HTTP/1.1 200 OK in all three scenarios. It’s only across the proxy with the HMAC authentication integrated that the 401 occurs.
Okay, I’ve narrowed it down to this statement equating to “false”:
(incomingBase64Signature.Equals(Convert.ToBase64String(signatureBytes), StringComparison.Ordinal));
This is only happening when I attempt to access the Web API through the reverse proxy. Locally (localhost) and intranet it equates to “true”. Any idea why these two values are different in this scenario?
Much appreciated!
Hi Anthony, Did you manage to resolve this issue? I am having the same issue when running locally (console app testing webapi).
Hey Phil, I just figured it out a couple hours ago.. Huzzah! My issue was that the clients requesting URI was different from where the Web API sees itself from because it was hosted on an intranet server accessed through a public-facing reverse proxy server.
The URI:
string requestUri = HttpUtility.UrlEncode(req.RequestUri.AbsoluteUri.ToLower());
…is one of the components used to generate the signature:
string signatureRawData = String.Format(“{0}{1}{2}{3}{4}{5}”, APPId, requestHttpMethod, requestUri, requestTimeStamp, nonce, requestContentBase64String);
…which is used to hash the API key for transit and decrypt it on the other end.
What I did to get around this is modify the requestUri string property used to generate the decrypting signature on the back-end Web API in to thinking it was hosted on the proxy:
string requestUriSub = requestUri.Substring(requestUri.IndexOf(“appwebsitename”)).Trim();
requestUri = “https%3a%2f%2freverseproxywebsitename%2f” + requestUriSub;
…Voila!
I hope that helps anyone else attempting to set this up behind a reverse proxy. Much thanks to Taiseer Joudeh for all the heavy lifting though, works like a charm!
Thanks Anthony for sharing your solution here, I’m sure it will help some one 🙂
Hello, I have the same problem. It is working fine in localhost, but it makes 401 Unauthorized error in the Azure. It seems the same Uri mismatch issue, but I’m not sure what is the solution for Azure. If any body knows the solution, please share it with me. Thanks in advance,
You said:
1. Client and server should generate the hash (signature) using the same hashing algorithm as well adhere to the same parameters order, any slight change including case sensitivity when implementing the hashing will result in totally different signature and all requests from the client to the server will get rejected. So be consistent and agree on how to generate the signature up front and in clear way.
2. In my opinion HMAC authentication is more complicated than OAuth 2.0 but in some situations you need to use it especially if you can’t use TLS, or when you are building HTTP service that will be consumed by terminals or devices which storing the API Key in it is fine.
So, if we want to create APIs for our client, we have to give them the library of the hashing algorithm? I mean, some devices with embedded controller (maybe website) need to consume the WebApi and they was using JavaScript or C++ or VisualBasic etc… The question is, our clients have to either write hashing algorithm (HMAC generation) for each language or the APIs provider must say that “please apply this algorithm along with secretkey to consume our APIs”.
I missed something? Could you please help me to expand my mind 🙂
Hi TAISEER,
Firstly thank you for your great post, it helps me about building a secure webapi for my integration project. But I wonder (and need help) how can I use HMAC system with javascript?
My client application has just native javascript and html codes and connected to back-end via ASPNET webapi. Before apply this HMAC solution I can easly get/post via API without auth. and secure.
Can you help me about this issue please.
Regards
Hi Emre,
Hmac Auth is not intended to be used with JS clients where they can not store the secret, you need to consider using OAuth 2, check my other posts about this topic.
Hi Taiseer,
Thank you for your fast response.
Hi TAISEER,
Firstly thank you for your great post.
Can you help me create key pair private/public. i don’t know how to create them. As you say APP Id can generate from a GUID, but when i change APP Id above by a other APP Id is not working.
Can you help me about this issue please.
Sorry I can’t get your question, if you changed the APP Id then you use the same APP Id with the client application too, the same applies for the secret.
Hi,
I have implemented as per guild lines.But you have mentioned client through program.cs file.I want to put request through ajax post.So please suggest me how can handle it.
You need to pass the value if the Authorization header using something like this. But keep in your mind that you need to do this with clients that can store the secret confidentially. If this is JS application then this is not the right way to go.
I love this! Thanks. What is the difference between an unauthorized response and a forbidden response? (Move the test client outside the company network and I get the “forbidden” response.)
Glad you find it useful, well Unauthorized means that you are not authenticated (Your username/password combination is in correct). I know it is misleading naming and should be unauthenticated, but this how is the header defined.
Forbidden means that you are authenticated but you do not have enough permissions to do this action.
Thanks! What do I have to do to permit your method to execute on an IIS server? Unprotected methods will run. Anonymous is enabled, no authentication scheme. Is there a step in the demo code I forgot?
Ah, well I may have found something, I didn’t decorate my method with the [RoutePrefix(…)] attribute . . .
I think, we can use Signature to replace Nonce (add to cached)
Thanks for this great well written article, I have implemented the authentication, I have tested this using a basic call with no payload or data posted.
I am not trying to use the same mechanism for file upload posted with some data (MimeMultipartContent) and the code is hanging when trying to compute the hash, ie calling: await httpContent.ReadAsByteArrayAsync()
I was just wondering if anyone has tried this before and could provide any insight to get this to work?
Hi Mark,
I guess you have solved this issue and you emailed the solution to me, right? It will be great if you shared the answer here once you have time so other readers will benefit from it. Thanks in advance.
Yes, I actually had 2 issues which I have resolved:
1) With regards to my previous post, the I tracked this down to the reading of the posted file stream more than once, calling:
request.Content.LoadIntoBufferAsync().Wait(cancellationToken);
in AuthenticateAsync did the trick and solved that.
2) When a request was not authorized, the result from the api contained the login page html in the api response.
I used this solution to solve that: http://slynetblog.blogspot.co.uk/2014/03/preventing-302-redirect-for-aspnet.html
Can you provide client example in javascript (angularjs). I don’t known how to do that. Please help me!
This will not work with JS apps, you can’t store the secret securely in JS.
I was able to get this to successfully run on my local environment using Visual Studio 2013 without using IIS. My machine is Windows 7 but when I moved the service out to a Windows Server 2008 machine (with .net framework 4.5 installed and selected for the app pool) the call from the client just hangs and eventually times out. If I comment out the HMAC attribute on my controller within the service, it works fine on this server. It’s almost like the filter process isn’t being recognized for some reason on the Windows Server 2008 machine?
Hi Leo,
I’m not sure if the OS will affect here, maybe different IIS version? Check the logs to see where the issue is exactly happening.
When I look at the web logs there isn’t even an entry for the request…it’s almost like its hanging up earlier than the logging. But again it works fine if I just comment out the hmac attribute..the iis version is 7..but it just feels like I’m missing something that supports the authentication filter within iis maybe? And again locally on my machine everything works fine using the hmac attribute…I’m trying to get this to production as soon as possible but I can’t get past this one…any other thoughts by any chance?
Just wanted to check in again to see if you had any thoughts…thanks!
Thanks for the detailed article and great examples. It would be nice if there was a spec for this type of access when OAuth flow does not make sense. The closest I found was a draft for Token Access Authentication which is similar to what you describe.
https://tools.ietf.org/html/draft-hammer-http-token-auth-01
Thanks for the great article. I was trying this in localhost with Authorized and Unauthorized requests and they all worked.
Then I deployed it to a server to do some experiments but this all the unauthorized requests are returning 200 (which must return 401). The same in localhost is acting correctly. By unauthorized requests, i mean the ones without any APPId or APIKey
Do you have any idea what might be the possible reason?
Sorry for the late reply, well I’m not sure what can be misconfigured here, can you trace the request using fiddler and compare it localhost.
Thanks for the response. It was probably my mistake somewhere in the build or deployment. However, I had a couple of more issues during the experiment and for all of them it ended up that I am having different versions of WebAPI in server and client.
Thanks for the great article again. I learnt very useful things.
You would only have WebAPI on the server though, right? What do you mean by WebAPI being different on the client?
Thank you for sharing such a high quality project with the community!
You are welcome Rich, glad to help 🙂
Hi Taiseer,
Its really great article for Api key authentication, I looked at angular js asp.net Identity authentication tutorial code. I used it for my security module and saved me lots effort
What Im trying to do is, I want key Api key in AspnetUser table in IdentityDB. same code need to use simpleprovider authentication and hmac authentication.
Is there are way I can place hmac authentication in simpleoproivderoauthauthentication use client id to differentiate client as angularjs or apikey.
Its gonna be really help with some suggesstions.
I really Like your article. very well structured.
well i tested using the same appid and appkey in my example
i have this key on the server side
A93reRTUJHsCuQSHR+L3GxqOJyDmQpCgps102ciuabc=
and in the client side: i used this 1
A93reRTUJHsCuQSHR+L3GxqOJyDmQpCgps102ciuabd=
the only difference is the last character from c to d.
i was expecting the validation to fail but went through.
Do you know if it s bad luck (the hashing returning the same for both) or something is wrong with the implementation somewhere (i cannot find any issue with it).
thanks
Hi Aymen,
I’m not sure where the issue is originating from, but any change on the key hashed value should result in 401, If I have time I will check it and let you know.
Wow finally a easy to follow example, thank you very much; this is brilliant
Thanks Andrew, happy to help!
Is it good idea to use HMAC authentication for entreprise? I mean: for instance company_1 wants to get data from company_2. So I can create web api for complany_2 with HMAC authentication and share format of signature, AppId and AppKey with compnay_1.
As long as the key is stored securely on server side then yes, as well you can think of OAuth client credentials flow as well.
Another question: do I need to have MD5 hash of request content if I use https? In case of POST requests content might have a big size and I’m worried about perfomance.
Hashing request body is not related to security, we just need to make sure that request body is not changed between requests, so even if you are using HTTPs you still need to hash request body.
Is it protection from “man in the middle”? By the way using oauth we don’t create content hash and just add access token to request. So why we need it for HMAC?
Hello Taiseer,
Your posts are awesome. My scenario is something like this: I have a web service that will expose some API. This API is built inside an ASP.NET web forms application. Now, the clients may be PHP, Java or ASP.NET. So, do I need to provide library for each platform to compute client side signature and send it to the web service? And how can I do that? I am not so pro in PHP and Java. I can only provide library for C#.
Also, please check this link: http://stackoverflow.com/questions/33056124/getting-method-not-allowed-error-when-using-asp-net-web-api-inside-asp-net-web. I am having some problem integrating your code with ASP.NET Web Forms.
I have answered you there, hope it will help.
How did you come up with 4d53bce03ec34c0a911182d4c228ee6c as your App ID and how does that pair with the Secret?
Nevermind, I can see that the AppID and AppKey are generated independently of one another and are only related in the allowedApps dictionary..
Good post, and thanks
You are welcome Joe.
Excellent blog Taiseer. I am following all of your post and all are helpful.
This HMAC authentication is fitting well in my case, but I have some doubt if you could get time to read through and help.
I am developing hybrid mobile app (using ionic framework) which talks to api (using asp.net web api). I would like to keep my app open and dont want my users to ask for register and login. But at the same time I can not keep my api open and that’s where I can use HMAC authentication which you explained well.
Only issue for me is, how to keep APP ID and SECRET secure in hybrid app. I need to ship my app with both pre-configured(either hard-coded in JavaScript file or using any db like sqlite). Anybody can download app on rooted phone and inspect inside app and easily gets hold of app id and secret and call my api.
so my question is,
1) Is it good practice to ship app with App Id and Secret? Or any other security flow required in addition to HMAC?
2) Do you think my concern is right about someone steal app id and secret and talk to my api and may trigger automated requests to get my server down? These days every body(facebook, googe,..) is giving open access to their api. How they deal with this situation?
3) if anything better you can suggest with my issue.
Thank you very much.
Hi Taiseer – Very nice example on this and nicely organized.
I understand the reason to hash the payload as part of the signature calculation, but is there a reason MD5 was chosen and not SHA256 to hash the payload? Reason I ask is the final signature will be the hash of all values using SHA256. I was comparing the similarities to AWS Signature V4 methodology and they use Hex(SHA256Hash() for the hashed payload.
Thank you.
Hi Adam,
As far as I remember, you can hash it SHA256 too, so there is no reason to use MD5 only.
Hello Taiseer,
first of all thanks for these great posts, for me it’s a big and reliable source of information.
I did setup an implementation of the Oauth which supports either cookie authentication as well as Bearer tokens.
I am using these two implementations by authenticating users from a web site as well as applications (mainly windows services and mobile apps) that uses a web api.
However some of my clients have some difficults to implement Bearer tokens to interface my API. They asked for an ApiKey implementation and this one seems to be very powerful.
Do you think I can mix both implementations in my web API project, that is, some clients authenticating by using bearer tokens and some others by using HMAC?
Hi Taiseer,
Great Post. Thanks for your effort.
Regards
Deepu
You are welcome, happy to help.
Your client side async call might cause a deadlock. How would you solve that?
Hmmm how did you decide that it might cause a deadlock? please elaborate more.
Thanks for the great post. If I would want to add restriction to some of the clients(via appid) in using the API (some may read others write or both) what would be the best approach ?
Hi Thomas, You are talking about scopes now, maybe you can take a look on this mature repo where the scopes might be supported, not sure 100%. Hope this help.
Hi Taiseer,
Thanks for the awesome explanation and the code repo!
I followed your implementation and deployed my Web Api project on Azure but I’m getting intermittent 404 errors. As weird as it may sound, the calls go through about half the time and the rest of the time, the HttpReponse comes back as a 404.
A request like “http://mywebapp.azurewebsites.net/api/foo/bar?val=123” sometimes ends up coming back as “http://mywebapp.azurewebsites.net/Account/Login?ReturnUrl=%2Fapi%2Ffoo%2Fbar%3Fval%3D123”, which explains the 404 but doesn’t explain why the above happens, and why it happens occasionally. It works fine on my local machine and I made sure to update the server time to my timezone so the time is synced.
Any thoughts? Thanks again!
It seems you used the default template for Web API project where cookie authentication is enabled, that is why you are redirected to the login view which doesn’t exist, please check how you can Web API from scratch by installing the needed NuGet packages and not using the default templates.
Hi there,
I have implemented the above, but something weird is happening when the client and server are located in different time zones.
In the isReplayRequest method, at this section “if ((serverTotalSeconds – requestTotalSeconds) > requestMaxAgeInSeconds)”
Sometimes the serverTotalSeconds is smaller than requestTotalSeconds. I’m not sure why but the subtraction gives a really large number, which does not make sense. These are ulong types, right. Here is an example:
In a random case where the serverTotalSeconds is equal to 1459382115 and the requestTotalSeconds is 1459382171, the subtraction gives 18446744073709551560. Although, the actual result, when i use a calculator, is -56.
This happens when subtracting ulong types.
How’s that possible?
Also, how come the server time is smaller than the request time?
Thanks.
Hi Saif,
Sorry for the late reply. Weird case to be honest. Did you find a solution for this so far?
Hi,
What i did was before performing the subtraction I always make sure the smallest value is being subtracted from the largest. A simple if statement and switching. But it’s weird. I’m sure both client and server clocks are accurate. I’m using UTC function in both machines to get the time. From my understanding it should really be ClientTime >= ServerTime.
Anyway. Let us know if you come up with a better solution.
Regards.
This is a great article, but I seem to have run into an odd problem that I can’t sort out.
My project is in VB.NET so I had to convert your code to VB.
The problem is on the client side CustomDelegatingHandler’s SendAsync on the line:
response = Await MyBase.SendAsync(request, cancellationToken)
When this line runs, response is not filled with any data, nor it is returned. The process just returns to the browser with no apparent action. The controller on the Web API returns a small amount of data, which I can see getting returned in Fiddler, but because the response object isn’t filled, the data is lost. I tried placing the line:
response As HttpResponseMessage = Await client.PostAsJsonAsync(apiBaseAddress & Convert.ToString(“api/Order”), JsonConvert.SerializeObject(td), CancellationToken.None)
In a Try/Catch block, but no error is thrown.
Any ideas?
Looks like I’ve figured it out. My project is not only VB.NET but it’s also a webforms project. The page was not async and that was why I was having the issues. I’ve got it all sorted out.
Again, great article.
You welcome, happy to know that you solved your own problem 🙂
Nice article!
It would be very nice if we can see the same implementation for MVC 6 (MVC + WebApi now together). I’m trying to do it but have issues since there a lot of changes comparing to WebApi 2.
Vladica
Thanks for your comment, I’m planning to start new series for ASP.NET core 1.0 very soon.
Awesome article. Thanks a lot for you time.
I had implemented this concept and works like a champ on HTTP, how ever when I test this on https on a farm that use a F5 it failed with unauthorized status.
Anyone has faced this one?
Hi Taiseer!
Thanks for really great article!
I have deeply dived into your implementation and met one problem. So i want to share it, maybe it’ll be usefull.
In very rare case serverTotalSeconds could be lesser then requestTotalSeconds (probably reason – issues with time sync) – so we have got underflow during this comparsion:
(serverTotalSeconds – requestTotalSeconds) > requestMaxAgeInSeconds)
and method returns TRUE (replay attack).
Real example:
serverTotalSeconds:1461328686, requestTotalSeconds:1461328697, requestMaxAgeInSeconds:300
Expected result from (serverTotalSeconds – requestTotalSeconds) are -11 (-11 is not bigger than 300 – OK, method result-FALSE) , but we’ve got underflow and result becomes 18446744073709551605.
18446744073709551605 > requestMaxAgeInSeconds, so method result – TRUE (replay attack, access denied)!
In my case(!) i have solved issue in this way :
//————————————————–
long serverRequestSubstraction = (long) (serverTotalSeconds – requestTotalSeconds) ;
if ( serverRequestSubstraction > (long) requestMaxAgeInSeconds) {
return true;
} //So, i am using long variable to store substraction result to prevent underflow
//————————————————–
Instead of:
//————————————————–
if ((serverTotalSeconds – requestTotalSeconds) > requestMaxAgeInSeconds)
{
return true;
}
//————————————————–
So, thanks a lot again, its very nice and usefull article, it helped me a lot!
You are welcome, and thank for your detailed explanation, I will revise the changes soon and will let you know if I have any comments.
Thank you for this nice explanation.
I had to change one thing though, because the code would only run when debugging.
So I changed:
var isValid = isValidRequest(req, APPId, incomingBase64Signature, nonce, requestTimeStamp);
To:
var isValid = Task.Run(() => isValidRequest(req, APPId, incomingBase64Signature, nonce, requestTimeStamp));
Great article, really helpful.
Any idea on how to parse and decrypt the authorization value?
This is really needed if you have client app that are sending values with 401 error response, so that you can parse the argument/values and tell where the issue comes from.
Hi Moses,
Currently do not have an example for this and I need to check how this is done, I will update you if I found something useful.
Great Post learn all new things regarding security
You are most welcome, thanks for your message.
Hey guys,
Did anybody managed to implement the client in PHP, I’m getting an unauthorised response from the API.
Not sure if I implemented this part all right:
C#
//Creating the raw signature string
string signatureRawData = String.Format(“{0}{1}{2}{3}{4}{5}”, APPId, requestHttpMethod, requestUri, requestTimeStamp, nonce, requestContentBase64String);
var secretKeyByteArray = Convert.FromBase64String(APIKey);
byte[] signature = Encoding.UTF8.GetBytes(signatureRawData);
using (HMACSHA256 hmac = new HMACSHA256(secretKeyByteArray))
{
byte[] signatureBytes = hmac.ComputeHash(signature);
string requestSignatureBase64String = Convert.ToBase64String(signatureBytes);
//Setting the values in the Authorization header using custom scheme (amx)
request.Headers.Authorization = new AuthenticationHeaderValue(“amx”, string.Format(“{0}:{1}:{2}:{3}”, APPId, requestSignatureBase64String, nonce, requestTimeStamp));
}
PHP
// Raw data
$signature = $api_id . $request_method . $request_uri . $request_timestamp . $nonce . $post_data;
$secret_key_byte_array = base64_decode($api_key);
// Calculate the hash
$hash = hash_hmac(‘sha256′, $signature, $secret_key_byte_array, true);
// Convert to base 64
$request_signature_base64 = base64_encode($hash);
$sig = $api_id .’:’. $request_signature_base64 .’:’. $nonce .’:’. $request_timestamp ;
// Headers
$headers = array(
‘Authorization: amx ‘ . $sig,
);
Does anybody have any pointers ??
Thanks,
Sorin
Hi Sorin, I have no experience in PHP but I hope some folks can you help here, try to post the question on SO, I think you will get a faster response.
Hi Sorin, do you have an example go how to call the client via php, this will be good to have as an example if customers don’t use .net. I too don’t know php.
I was reading the article, it’s very helpful and well organized.
Then I saw the author, So I decided to keep a comment, how the world is small.
Hope to meet again my friend.
Ohhh.. Rick, good to meet you “virtually” my friend, thanks for your lovely comment, I will call you If I visited Beirut soon 🙂
Hi Taiseer,
Thanks for your code one again, we still use this will great success. I have however found a small anomaly when using it..
To date we have have used this to successfully authenticate apis between a few applications, which coincidentally the client and server parts have all been running on the same server.
We have now exposed an API to be called remotely where all the calls have not been authorised. I have tracked this down to the timestamp. The Server time is a few seconds behind the client time, if I subtract a couple of minutes at the time of the request, it all works. I assume this is all we can do really?
Hi Taiseer,
I was trying to use the sample in UWP windows 10 app and I have converted signature signing method like the following from the client app (mobile app) for MD5 hashing and HMAC256. However, for some reason from server side which is web api, I am unable to match the signature.
ComputeMD5
==========
private static string ComputeHash(string content)
{
var ibuffer = CryptographicBuffer.ConvertStringToBinary(content,BinaryStringEncoding.Utf8);
var hashed = HashAlgorithmProvider.OpenAlgorithm(HashAlgorithmNames.Md5).HashData(ibuffer);
return CryptographicBuffer.EncodeToBase64String(hashed);
}
HMAC256
=======
private static string CreateHMAC256(string hashKey, string messageToSignIn)
{
var macAlgorithmProvider = MacAlgorithmProvider.OpenAlgorithm(MacAlgorithmNames.HmacSha256);
var encoding = BinaryStringEncoding.Utf8;
var messageBuffer = CryptographicBuffer.ConvertStringToBinary(messageToSignIn, encoding);
IBuffer keyBuffer = CryptographicBuffer.ConvertStringToBinary(hashKey, encoding);
var hmacKey = macAlgorithmProvider.CreateKey(keyBuffer);
IBuffer signedMessage = CryptographicEngine.Sign(hmacKey, messageBuffer);
return CryptographicBuffer.EncodeToHexString(signedMessage);
}
Below line of code was not matching incoming signature and MD5 generated signature. Could you please suggest what I am doing wrong.
using (var hmac = new HMACSHA256(Convert.FromBase64String(sharedkey)))
{
return (inSignature.Equals(Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(data))), StringComparison.Ordinal));
}
Thank you
Deepu
Hello Deepu,
I do not have specific answer for this, I need to do small POC to test it out, hopefully I will find some spare time during this weekend.
Thank you so much Taiseer, That would be awesome !!!
Excellent article. I did want to point out two items that I ran into while implementing client side for this setup on a different platform. Posting here in case anyone else runs into same issue.
1) The URLEncode function above is generating lowercase %xx hex values, need to make sure that your client libraries do the same. In my case, uri_escape with perl was generating them in uppercase. Explicitly lowercasing the encoded request uri in the example would make this failsafe.
2) The hmac hash function above is returning the base64 string with padding (trailing equal sign) – need to make sure that your library does the same. For perl, Digest::SHA’s hmac_sha256_base64 did not, but using hmac_sha256 passed through encode_base64 worked fine.
Nathan, thanks for sharing your feedback. I’m sure it will be useful for someone want to create client in perl.
So, I just want anyone who finds this like I did to know what a possibly issue might be.
The ToLower() needs to be AFTER the UrlEncode. Like this:
string requestUri = System.Net.WebUtility.UrlEncode(request.RequestUri.AbsoluteUri.ToLower()).ToLower();
It took me a bit of time to realize this was my issue because in my code I saw the AbsoluteUri.ToLower() and thought that would make the encoded URI lower case but the individual elements i.e. %2E are not %2e unless you ToLower() the resultant string.
Hai Sir,
I read the article and download the source code and run it on my PC. But the issue is that It will authenticate only the Post method in the controller.
My question is that
1 What changes are made to authenticate GET/DELETE methods also
2 Here have the code for authenticate api/orders controller methods only.What changes are made in code to get authenticate methods from other controllers?
Waiting for yours valuable reply.
Hello,
Please refer to this Hawk authentication for a complete and tested method to implement HMAC authentication.
Is it possible to use HMAC authentication over SSL (https) ?
Yes, you can use it over https for sure, it is more secure.
How to fix An unhandled exception of type ‘System.AggregateException’ occurred in mscorlib.dll exception in RunAsync().Wait()
How if type ‘RoutePrefix’ and ‘IHttpActionResult’ could not be found? And how to use HMAC authentication over HTTPS?
Just a note to anyone who reads the article and also happens to read the comments – the code assumes the client’s clock is perfectly synced with the server clock, and this is rarely the case.
To get around this, instead of converting the time stamps to unsigned longs, convert them to longs and then do a Math.Abs() call on the serverTime – requestTime so the window to check is both before and after the server time.
Marlon, thanks for sharing this. You are correct we should take care of time skew between client/server.
Hello Taiseer – I have setup the HMAC as described in the article and the github source. However in my web application , I can see firing the delegatinghandler, calling the api, the api processing the request and returing the response from ResultWithChallenge.ExecuteAsync().
However on my client (web app), the last line – response.Result.IsSuccessStatusCode, does not hit the breakpoint. The browser just keeps spinning. I dont have any other delegating handlers or filters.
var response = client.GetAsync(baseurl + endpoint);
response.Wait();
if (!response.Result.IsSuccessStatusCode) return null;
On further analysis, this is happening only in a web application and not in a console application. Is it running in a deadlock in Custom DelegatingHandler at response = await base.SendAsync(request, cancellationToken); ?
I have the same Issue.
Hi Mayuresh, Did you handled this issue? From webapplication custom delegate never returns. i.e Custom DelegatingHandler at response = await base.SendAsync(request, cancellationToken);
when an android application not in debug mode generates a io exception: com.android.volley.NoConnectionError: java.io.IOException: No authentication challenges found error throw. Please look in this issue
Hello Krshna,
The comment is irrelevant for the post, Sorry for not being able to help. Post your issue to Stack overflow to get a faster response.
After copying the example word for word I am unable to get any response from the server.
var content = await httpContent.ReadAsByteArrayAsync() line is hit, then nothing, no error.
this stackoverflow post touches on my issue http://stackoverflow.com/questions/38904954/httpcontent-readasbytearrayasync-fails-without-error-inside-delegatinghandler
But using the answers code
var ms = new MemoryStream();
await httpContent.CopyToAsync(ms);
ms.Seek(0, SeekOrigin.Begin);
var content = ms.ToArray();
Always returns null to the client.
Can you try to form the Repo and try the sample. I’m afraid there is something different when you tried to regenerate it. Check the NuGet versions as well in case there is any breaking changes between your versions and the one I used.
Hi,
This is what i was looking for. I have one question, my client is angular app. So, i can create the signature using “hmac-sha256 javascript” right ? and then i can process it to server. I am a noob. So, please tell me if i am going in right direction.
Hi Jawand,
You can not use this security with client built using JS, those are nonconfidential clients, where are you going to store the secret security on the App? You need to use token based authentication for Angular App, check the post here
Thanks for your reply,
I am already using Token based authentication “bearer and refresh” for user . The thing my boss wants is to secure our API like Google do, before allowing any one to use there API they gave us clientId and ClientSecret.
That’s what i am looking for. I know this can be done using your article for Mobile and Console Applications. But i want the same thing to be used for Web Client (JS). I hope you get what i am looking for and tell me if i am doing something wrong here.
and here is another thing my boss said we are not sure if client is going to use SSL. So, he said we need to secure the plain text password over HTTP. and again if i go for javascript SHA256 any one can sneak into client code and figure some way to decrypt the credentials being sent over HTTP. and also there is danger of man in the middle attack. to get the HASH and in some way a evil guy can figure his way out.
Thanks
Hi Taiseer – its really a great approach to authenticate web apis i followed it and it was a successful try. One thing i have been confusing about that is on the client side signature data was generated using APIKey while on the server side APPId is being used to generate and compare signatures so this is where am confused a little bit so can u please give a hint about it, it would be really helpful.. Thanks
Hi Ankit,
The Api key should be used on the client and server to generate the signature, the AppId is used to get the ApiKey from the permanent storage only.
Thanks Taiseer for your help I think its more clear now Thank you very much
hi
i have two question
1- do always after AuthenticateAsync method , ChallengeAsync method is run?
2- how can i redirect user to some Action in AuthenticateAsync method
thanks alot
Hello Thanks for the great article. one question how about server application generates apikeys for many users
1. do we need to have many appIds for each users?
2. How to generate appId
3. Should client and server have to have both keys on each side?
I’m really new to this pelase explain me
thanks
Hello Gayan,
I recommend you to read the post one more time, all those questions have been answered in the post.
Regarding generating the AppId, you can use whatever unique identifier you prefer, in my case I use GUIDs
Thank you very much Taiseer Joudeh if you can do the same implementation on Asp.Net Core WebApi that would be great help thanks
You are most welcome Gayan, happy to help always 🙂
I have implemented on asp.net Core – http://bit.ly/2AybLYl
Comments are welcomed.
Thanks RredCat
Excellent article. I have implemented a very similar one (using yours as reference). Works beautifully.
Excellent Article. Explains all about topic in detail.
One question – Is it required to use a “Unix Time”? Can’t we simply use date time stamp in UTC format to achieve the same results?
Hello J,
Unit time is more standard way to achieve this, I never saw an implementation using UTC.
Thanks for this but is it not really important to re-use the httpclient instead of creating a new one each time? So shouldn’t the header be added to the request and not the client?
Hi Taiseer,
Excellent article, thanks a ton for sharing. I just wanted to know is there any copyright/licensing issue if we refer this code in our application. We’re from a vendor company and we’d like to use this for an ecommerce application, we’re developing for one of our client.
With Warm Regards,
Parimal Modi
Hi Parimal,
Please feel free to use the code as-is or modify it to meet your needs.
Thanks Taiseer
I love this article, But can you show me how to configure swashbuckle for it?
Thanks for the article, its been very very helpful :). I got it working just fine for authorised api calls.
But im not getting a 401 on an unauthorised API call, instead its redirecting to my login page
I believe this is because the api is part of a larger MVC site.
The only way i can force a 401 is by checking for the /api url and an unsuccesful status code in “Application_EndRequest” in Global.asax.cs: (an unauthorised api call has a staus code of 302 as its redirecting to the login page)
protected void Application_EndRequest(Object sender, EventArgs e)
{
HttpContext context = HttpContext.Current;
if(context.Request.Url.AbsoluteUri.Contains(“/api”))
{
if (!context.Response.Status.Substring(0, 3).Equals(“200”))
{
context.Response.ClearContent();
context.Response.Write(“Wrong credentials”);
context.Response.StatusCode = 401;
}
}
}
The above is not ideal as all MVC and api calls will end up in the Application_EndRequest, if anyone has a better idea i would be gratefull for some assistance?
How would i call the webapi using postman. How would i generate the token or do i have to create a program to generate authorization signature for me?
It possible to call in the java script? Because you sample code is in the console.
Excellent article! Covers almost everything, preventing redudant requests was really great.
Thankyou so much for sharing.
I am really loving the theme/design of your website.
Do you ever run into any browser compatibility problems?
A small number of my blog audience have complained about my
site not operating correctly in Explorer but looks great in Firefox.
Do you have any ideas to help fix this issue?
Great article.
It helped me alot.
For now I can authenticate every request on my API, but I don’t now how to pass the parameters (ie: token, appID or nonce) to the API’s final method.
I can get the authentication header encrypted, but I would like to have it unecrypted, just for not to do a double job and decrypt the header again.
Thank you.
It’s awesomе to pay a visit this site аnd reading the vieᴡs
of all mates concerning this article, while I am also eager of
getting familiarity.
Excellent goods from you, man. I’ve have in mind your stuff prior to and you’re just too great.
I actually like what you have obtained here, certainly like what you’re
saying and the way in which by which you assert it. You make it entertaining and you continue to take care of to keep it smart.
I can’t wait to learn much more from you. That is actually a terrific website.
Take a look at our website for the best friv games online. Multiple kinds of the best
friv games and is actually all free.
Great post! Have nice day ! 🙂 tovhc
Hi Taiseer Joudeh, that’s good article I have implemented in ASP.NET Web API With Web APP MVC . Working fine . but Now I want to Test on POSTMAN ,SWAGGER Tools and Also want to give client to test API’s by using given APP ID and API Key so how to proceed it. I mean in Header how to using these information. Please guide me in this regards.
Thanks
What if I don’t want to use a bew HMACAuthorize attribute on my c9ntroller methods?
I wanr to use the standard Authorize attrubute, and be able to authorize EITHER with a bearer token OR an api key. Is that posssible, and how?
Otherwise I must make a clone of every controller and use the HMACAuthorize attribute on the clone, and that’s an awful lot of redundancy as there are over 40 controllers in this OAuth2 API that I am tasked to adapt to being able to be called from another API as well as the already existing authorization..
Thanks!
It possible to call in the web api? Because you sample code is in the console.
Can Any one help me.
Hello excellent article, it’s working perfect but I’m having issues sending Authorization from WebAPI client, because I want to debug WebAPI and using console I can’t.
I’m using this DHC client: chrome-extension://aejoelaoggembcahagimdiliamlcdmfm/restlet_client.html and sending this RAW header:
Authorization: {amx 4d53bce03ec34c0a911182d4c228ee6c:Ujpil+Z5V6uEHsXZL0o+fIpsNiGeFoDPABrOd1pnomw=:7b3d1f457e18466eac0c9c5dade46aec:1622772357}
I always receive a 401 response (Unauthorized). How could I achieve this?
Let me know where I can send a picture so you can help with this!
Thanks!!
Thanks for a GREAT tutorials and sample code. I had a question regarding the code.
In the below code why are we using the Unix time (epochStart) ?. Can’t we just use the UTCNow time instead of subtracting from epochStart ?
DateTime epochStart = new DateTime(1970, 01, 01, 0, 0, 0, 0, DateTimeKind.Utc);
TimeSpan currentTs = DateTime.UtcNow – epochStart;
var serverTotalSeconds = Convert.ToUInt64(currentTs.TotalSeconds);
var requestTotalSeconds = Convert.ToUInt64(requestTimeStamp);
if ((serverTotalSeconds – requestTotalSeconds) > requestMaxAgeInSeconds)
{
return true;
}
Dear Taiseer Joudeh,
I like your demos and excellent demonstrations.
I’m having an issue regarding above Post.
I always get Un-Authorized when i use my generated key (API Key ) i.e. generated by the method you provided.
whats the possible reason behing Un-Authorization.
Note : I replaced client side API key as well as in HmacAuthorizationProvider.
Thanks in advance.
I read through various types of authentication which Web API supports e.g Basic Authentication, Oauth 1.0 and Oauth 2.0 etc. I am going to develop an API where I have to transfer PDF files which will hold very sensitive data. I am confused to decide on which authentication mechanism to follow after reading HMAC authentication. Is it good to use HMAC with OAuth 2.0 as Oauth 2.0 is little easy to implement? Can you suggest?
Thank your for the decent post and sample code!
You cached the nonce value without a client identifier, which means it must be unique to all clients. You probably want to attach the app ID to it so that two different clients can use the same nonce in a timespan.
Hi,
Thanks for the article! I was playing around with the code sample, and I have tried out a situation where the client has the API Key as “A93reRTUJHsCuQSHR+L3TxqOJyDmQpCgps102ciuabc=” and the service has the API Key as “A93reRTUJHsCuQSHR+L3TxqOJyDmQpCgps102ciuabe=” . Notice the last character before the = sign is different in both the keys. Surprisingly the service according to the code above has given a result that the request was valid, even though the API key is technically incorrect. Could you please shed some light what is going on there?
Thanks!
Hello, very good article.
Is it possible to implement it in wcf rest?
Awesome post. Implemented successfully. Thanks!
Awesome article. Thanks for sharing
I am facing one issue which might be weird. In the below condition
if ((serverTotalSeconds – requestTotalSeconds) > requestMaxAgeInSeconds)
{
return true;
}
Some time my requestTotalSeconds has greater value than serverTotalSeconds . I am using same code in both applications to generate the time stamp.
Please suggest something for this.
Thanks for the Article , but I am facing a challenge , after running the code it works on my system but after I host the Api and try to run the client from another system it is showing me this error “Failed to call the API. HTTP Status: Unauthorized, Reason Unauthorized” your response will be appreciated thenks
Thank for this great inspiration.
Please how do i pass get request url when consuming api via xamarin android app.
hi Taiseer Joudeh,
I’m working on .NET framework, could you please share me a post related to Cloud/Service Oriented Architecture (SOA) ?
Many thanks
Thai
Hi TaiSeer, you did a great job. I think this sulotion is best suite for the case of IOT device without interacting with user, and communicating with HTTP. And you gen bearer token with each single request by hash the request body. So the token is just used for one time! The nonce and timestamp is for replay attack.
While it is like JWT somehow, JWT token requires HTTPS.
What changes need to be made for this to work using .net 5 or .net 6 ?