Mandrill REST API

OK, here’s another REST API example. In the same system I blogged about yesterday, I also had to use the Mandrill API to send out some transactional emails. Mandrill is a relatively new service from MailChimp that can be used to send out e-mail via simple API calls. It’s pretty nice, and free to use, up to a certain point.

You can actually use it via regular old SMTP, with some control info added into the headers, or you can go ahead and use their REST API. I decided to use the API. Initially, I had no luck with that at all. After thrashing around a bit, I eventually figured out that I had to update to the newest version of EasyHttp. Then, it started working fine, no problem. I didn’t bother looking into the internals of EasyHttp to try and figure out what was causing the problem. I’m kind of curious, but, as usual, it’s more important to just get the thing working.

So, here’s another Gist with some code:

/*
 * Code/Mandrill.cs
 * These are calls to the Mandrill email service.
 * Reference: https://mandrillapp.com/api/docs/
 * 
 * ajh 2012-08-09: new
 */
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Configuration;
using System.Dynamic;
using EasyHttp.Http;
using System.Net;
using log4net;

namespace Sample.Web
{
    enum MandrillError
    {
        OK,
        WebException,
        HttpNotOk,
        Invalid,
        Rejected,
        Unknown
    }

    public class Mandrill
    {
        static string MandrillBaseUrl = ConfigurationManager.AppSettings["MandrillBaseUrl"];
        static Guid MandrillKey = new Guid(ConfigurationManager.AppSettings["MandrillKey"]);

        public static bool SendActivationEMail(BLL.TrialSignup ts, out string errorMsg)
        {
            string activationLink =
                HttpContext.Current.Request.Url.GetLeftPart(UriPartial.Authority) + "/Register/Activation.aspx?id=" + ts.Id;
            
            //send-template(string key, string template_name, array template_content, struct message) 
            dynamic sendParams = new ExpandoObject();
            sendParams.key = MandrillKey;
            sendParams.template_name = "Secret Project Trial Activation";

            sendParams.template_content = new List();
            
            sendParams.message = new ExpandoObject();
            sendParams.message.subject = "Here's your Secret Project activation email";
            sendParams.message.from_email = "info@SecretProject.com";
            sendParams.message.from_name = "Secret Project";

            sendParams.message.to = new List();
            sendParams.message.to.Add(new ExpandoObject());
            sendParams.message.to[0].email = ts.EMail;
            sendParams.message.to[0].name = ts.Name;

            sendParams.message.track_opens = true;
            //sendParams.message.track_clicks = true;

            sendParams.message.global_merge_vars = new List();
            sendParams.message.global_merge_vars.Add(new ExpandoObject());
            sendParams.message.global_merge_vars[0].name = "NAME";
            sendParams.message.global_merge_vars[0].content = ts.Name;

            sendParams.message.global_merge_vars.Add(new ExpandoObject());
            sendParams.message.global_merge_vars[1].name = "LINK";
            sendParams.message.global_merge_vars[1].content = activationLink;

            errorMsg = string.Empty;
            
            MandrillError merr = SendMessage(sendParams);

            switch (merr)
            {
                case MandrillError.OK:
                    return true;
                case MandrillError.WebException:
                case MandrillError.HttpNotOk:
                    errorMsg = "There was an issue sending your activation e-mail. Please try again later or call us directly.";
                    break;
                case MandrillError.Invalid:
                    errorMsg = "Your email address appears to be invalid. Please try again with a valid address, or call us directly.";
                    break;
                case MandrillError.Rejected:
                    errorMsg = "Your activation email was rejected. Please try again with a valid address, or call us directly.";
                    break;
                case MandrillError.Unknown:
                    errorMsg = "There was an unknown problem sending your activation email. Please try again, or call us directly.";
                    break;
            }
            return false;
        }

        public static bool SendSalesNotification(BLL.TrialSignup ts)
        {
            dynamic sendParams = new ExpandoObject();
            sendParams.key = MandrillKey;
            sendParams.template_name = "Secret Project Trial Sales Notification";

            sendParams.template_content = new List();

            sendParams.message = new ExpandoObject();
            sendParams.message.subject = "Secret Project Trial Account Notification";
            sendParams.message.from_email = "info@SecretProject.com";
            sendParams.message.from_name = "Secret Project";

            sendParams.message.to = new List();
            sendParams.message.to.Add(new ExpandoObject());
            sendParams.message.to[0].email = ConfigurationManager.AppSettings["SalesEmail"];
            sendParams.message.to[0].name = "Secret Project Sales";

            //sendParams.message.track_opens = true;
            //sendParams.message.track_clicks = true;

            sendParams.message.global_merge_vars = new List();
            sendParams.message.global_merge_vars.Add(new ExpandoObject());
            sendParams.message.global_merge_vars[0].name = "NAME";
            sendParams.message.global_merge_vars[0].content = ts.Name;

            sendParams.message.global_merge_vars.Add(new ExpandoObject());
            sendParams.message.global_merge_vars[1].name = "COMPANY";
            sendParams.message.global_merge_vars[1].content = ts.CompanyName;
            
            sendParams.message.global_merge_vars.Add(new ExpandoObject());
            sendParams.message.global_merge_vars[2].name = "EMAIL";
            sendParams.message.global_merge_vars[2].content = ts.EMail;

            MandrillError merr = SendMessage(sendParams);

            switch (merr)
            {
                case MandrillError.OK:
                    return true;
                case MandrillError.WebException:
                case MandrillError.HttpNotOk:
                case MandrillError.Invalid:
                case MandrillError.Rejected:
                case MandrillError.Unknown:
                    break;
            }
            return false;
        }

        private static MandrillError SendMessage(dynamic sendParams)
        {
            ILog _log = log4net.LogManager.GetLogger("Mandrill/SendMessage");

            string url = MandrillBaseUrl + "/messages/send-template.json";

            var http = new HttpClient
            {
                Request = { Accept = HttpContentTypes.ApplicationJson }
            };

            EasyHttp.Http.HttpResponse response;
            try
            {
                response = http.Post(url, sendParams, HttpContentTypes.ApplicationJson);
            }
            catch (WebException ex)
            {
                _log.ErrorFormat("Error: WebException - {0}", ex.Message);
                return MandrillError.WebException;
            }

            if (response.StatusCode != HttpStatusCode.OK)
            {
                _log.InfoFormat("Response = {0} - {1}", response.StatusCode, response.StatusDescription);
                _log.Info(response.RawText);
                return MandrillError.HttpNotOk;
            }

            dynamic rv = response.DynamicBody;
            _log.InfoFormat("email: {0}, status: {1}", rv[0].email, rv[0].status);

            string send_status = rv[0].status;
            if (send_status == "sent" || send_status == "queued")
                return MandrillError.OK;

            // otherwise, it should be "rejected" or "invalid"
            if (send_status == "invalid")
            {
                return MandrillError.Invalid;
            }
            if (send_status == "rejected")
            {
                return MandrillError.Rejected;
            }

            // unexpected...
            return MandrillError.Unknown;
        }
        
    }
}

I hope someone finds it useful and/or entertaining.
In this code, I’m sending out two e-mails, one is a trial account activation email, and the other is just a notification to sales. On the first, I’m looking at the errors closely and trying to return a useful error message, so the customer knows that something has gone wrong and has a clue about how to handle it. On the second, I’m less worried about that, as it’s just a notification to our salesperson.

Zoho CRM REST API

I have about a dozen programming-related items floating around in my head that I’d like to write up short blog entries about. Maybe I’ll manage to get one or two written up tonight.

I’ve been doing some work recently with some random REST APIs. When I’m calling REST APIs from .NET, I like to use EasyHttp. Based on the stats on its NuGet page, I guess it’s not that popular, but it’s updated often enough, and it works well for me.

I recently worked on a project where I had to create a lead in Zoho CRM, using their API. I had a little bit of trouble, but only because I didn’t have quite the right parameters set — it turns out that “newformat=1” is really important. I had looked for some sample C# code for the Zoho API when I started this, and didn’t find any, so I thought I’d post a quick Gist with my code:

/*
 * Sample C# code to create a lead with the Zoho CRM API.
 * ajh 2012-08-23
 */
using System;
using System.Collections.Generic;
using System.Configuration;
using System.Dynamic;
using System.Linq;
using System.Net;
using System.Web;
using System.Xml;
using System.Xml.Linq;
using EasyHttp.Http;
using log4net;

namespace Sample.Web
{
    public class ZohoCrm
    {
        static string ZohoApiBaseUrl = "https://crm.zoho.com/crm/private/xml/Leads/insertRecords"; 
        static string ZohoApiKey = ConfigurationManager.AppSettings["ZohoApiKey"];

        public static bool CreateLead(BLL.SampleLead sl)
        {
            ILog _log = log4net.LogManager.GetLogger("ZohoCrm/CreateLead");

            XDocument xmlData = new XDocument(
            new XElement("Leads",
                new XElement("row", new XAttribute("no", "1"),
                    new XElement("FL", new XAttribute("val", "Lead Source"), "Trial Signup"),
                    new XElement("FL", new XAttribute("val", "Company"), sl.CompanyName),
                    new XElement("FL", new XAttribute("val", "Last Name"), sl.Name),
                    new XElement("FL", new XAttribute("val", "Email"), sl.EMail)
                )));

            string url = string.Format("{0}?authtoken={1}&scope=crmapi&newFormat=1&xmlData={2}", 
				ZohoApiBaseUrl, ZohoApiKey, 
                HttpUtility.UrlEncode(xmlData.ToString()));

            var http = new HttpClient
            {
                Request = { Accept = HttpContentTypes.ApplicationXml }
            };

            dynamic emptyPost = new ExpandoObject();            
            EasyHttp.Http.HttpResponse response;
            try
            {
                response = http.Post(url, emptyPost, HttpContentTypes.ApplicationXml);
            }
            catch (WebException ex)
            {
                _log.ErrorFormat("Error: WebException - {0}", ex.Message);
                return false;
            }

            if (response.StatusCode != HttpStatusCode.OK)
            {
                _log.InfoFormat("Response = {0} - {1}", response.StatusCode, response.StatusDescription);
                _log.Info(response.RawText);
                return false;
            }

            XDocument xdoc;
            try
            {
                xdoc = XDocument.Parse(response.RawText);
            }
            catch (XmlException ex)
            {
                _log.ErrorFormat("Error: XmlException parsing API response - {0}", ex.Message);
                _log.Info(response.RawText);
                return false;
            }

            string msg;
            try
            {
                msg = xdoc.Descendants("result").First().Element("message").Value;
            }
            catch (Exception ex)
            {
                _log.ErrorFormat("Error: Exception reading from API response - {0}", ex.Message);
                _log.Info(response.RawText);
                return false;
            }
            if (msg != "Record(s) added successfully")
            {
                _log.InfoFormat("Unexpected XML result: {0}", msg);
                _log.Info(response.RawText);
                return false;
            }

            return true;
        }
    }

(See also this Gist.)

I’m also using log4net here.  I’m a really big fan of log4net, and I use it in almost all of my projects. I have a pretty standard log.config file I use that sets up a RollingFileAppender, with ten files of 100k each. That’s usually enough to keep a few days or weeks worth of history, depending on the level of logging and the level of activity. And I never have to worry about the log files growing out of control; they just keep rolling over. I think log4net is pretty popular, based on the numbers on their NuGet page.

I try to do a lot of error-handling whenever I’m dealing with a REST API. There are plenty of things that can go wrong. You can see in the code above that I try to trap anything that might throw an exception and log it. I’m never sure if I’m doing this the “right” way. I know some people, when writing a routine like this, would just let the exceptions happen, then have a try/catch around the call to the CreateLead method, and deal with it at a higher level.

Classic ASP – SQL Injection

We still have a few old sites at work that are in classic ASP. One of the problems that tends to occur on these old sites is SQL injection attacks. A lot of old ASP code was written without taking SQL injection into account. It would be great if we could just rewrite all of these sites in ASP.NET, but sometimes the client isn’t interested in doing that.

Well, yesterday, we had a production ASP site get hit. This is a site for an old client that I’ve personally never worked on, so I really knew nothing about it, but I’m getting dragged in, now that we need to clean it up. Looking at the site, I’m actually surprised this hadn’t happened earlier.

Looking at the code, I see a few places where we could be doing a better job of input validation. And also a few places where we’re doing un-parameterized SQL, which is a big no-no if you want to avoid SQL injection. So, I’m going to try to clean some of that up.

I also want to use URLScan on the server, with some SQL injection rules, to try to get some of this stuff caught at the IIS level. I found this article on how to add some rules to the URLScan.ini file to mitigate SQL injection attacks. (I actually first started reading this article, but then remembered that this particular web server is still on IIS 6.)

When I started poking around on the server, I was surprised to see tha URLScan was already installed. However, it was not configured to do any SQL injection prevention. So, Monday morning, I’m going to try to add the SQL injection rules to the ini file, and see if that breaks anything. Then, I hope to have time to tighten up the code a bit and roll out a new version. I can’t say I’m excited to be working on nearly decade-old VBScript, but hey, it’ll be good for a few laughs, right?

Using SSH with Mercurial

We’ve been using Mercurial and Bitbucket at work for quite a while now. Things have been going pretty smoothly, and I don’t regret the choice of Hg over Git or TFS at all. Nor do I regret using Bitbucket as a back-end. They’ve had occasional outages, but probably no more than we’d have with Github or any other web-based service.

I had a bit of a problem pushing a really big changeset today though, so I decided to switch from HTTPS to SSH. It took about 20 minutes to set up, following these instructions. And I had to look here for some additional information on Pageant. I wish there had been a smoother, faster, way to get the whole thing set up, but it’s running smoothly now that I’ve got it all figured out.

PHP

I’ve been doing enough PHP work lately that this blog post really speaks to me. Here’s a good quote:

PHP isn’t so much a language as a random collection of arbitrary stuff, a virtual explosion at the keyword and function factory.

I kind of like Drupal, which is of course written in PHP, but I think I’d like it more if it was written in… something else. I’m not sure what.

You can certainly write good things in PHP, and Drupal is an example of that. CodeIgniter seems like a good thing too,  though I haven’t had much experience with it.

I recently had to do some work on what I’d call a “legacy” PHP site, which was basically all PHP spaghetti code, poorly done HTML, and questionable JavaScript. I wonder how many sites like that one are out there, written 5 or 10 years ago, possibly by an amateur, and working *just* well enough that nobody wants to pay to rewrite it from scratch. Probably a lot!

Stack Overflow to the rescue

Don’t you love it when a completely random person on StackOverflow, with a reputation score of just 1, has the answer you need to an annoying problem that has completely screwed up a production rollout? And which you would never have figured out on your own? Thanks!

ASP.NET AJAX Control Toolkit bugs

We make fairly extensive use of the AJAX Control Toolkit at work. When I first started at my current job, I wasn’t quite sure about it. In practice, most of the stuff we used seemed a little buggy and kind of kludgey. As I learned more about it, though, I started liking it. When used correctly, it’s quite useful, and holds up well, in terms of cross-browser compatibility and stuff like that.

However, once in a while, I stumble across an annoying bug. The most recent one cropped up after upgrading to a newer version of the ACT on a client site. I’m always a bit leery about upgrading the ACT, since it does often expose bugs that weren’t present in the prior version, and that don’t surface in casual testing.

In this case, we encountered a bug that occurs when you have a ValidationSummary control and a ModalPopupExtender on the same page.

If you Google the problem, you will find multiple mentions of it on StackOverflow and on the ACT bug tracker on CodePlex. The solution, basically, is to inject a script with just a semicolon in it, right after the ValidationSummary. (See here for a simple explanation.)

There are indications that the bug has been fixed, but then other indications that it re-surfaced again later. I can tell you that I was using the latest ASP.NET 3.5 version of the ACT at the time, and it’s definitely still a problem.

I don’t necessarily mind having to use these kinds of workarounds. These days, it’s fairly easy to Google an error message, and find multiple links to your problem on StackOverflow and in CodePlex bug reports, and in various other places. (I’m old enough to remember the days when we didn’t have Google or StackOverflow, so I don’t take these things for granted!) But, I would really like to encourage anyone reading this to please do one little thing for me: If you’re implementing a non-obvious workaround like this in your code, please document it with an explanation and/or a link back to the bug report or StackOverflow page. There’s nothing quite like having to do maintenance programming on a system created by someone else, finding something like this workaround on the page, undocumented, and trying to figure out what the purpose of it was.

I think that this is a problem that plagues many ASP.NET and PHP systems, especially ones that were developed with fairly loose coding standards, no peer review, and that have been around for a few years. There are often oddball workarounds in the JavaScript, CSS, and server-side code that aren’t documented and that are often fragile if tampered with. Every time I have to do maintenance programming on a system like this, I try to leave it looking at least a little bit better than it was when I started. If I can remove some unnecessary workarounds, change some non-obvious variable names to useful ones, and add some comments to explain what’s going on, I try to do that.

Speaking of the ACT, I see that a new version was released this month. Stephen Walther has been doing a lot of work on the ACT lately. This most recent release includes major revisions to (or perhaps a complete rewrite or) the file upload control. Trying to get file upload functionality working nicely on a page that’s doing a lot of tricky async stuff is pretty difficult. I used the older ACT file upload control on a page recently, and it works OK, but it required a few… workarounds. (All of which I tried to document, of course.) I don’t think that this new one will require fewer workarounds, since the basic limitations inherent in doing a file upload from a browser will still exist, but at least it will allow for some nice feedback through HTML5 upload progress events, as mentioned in the blog post.

ASP.NET ListView notes

I jump around between projects a lot, and I often don’t get deep enough into any particular thing to really learn it inside and out. I got a chance to work on a fairly complicated ASP.NET page recently, though, and I picked up a few new things.

The approach I took with this page, since it had a number of independent parts, was to try to keep them isolated and only spin things up when needed. So, in one panel, where I had a ListView, I wanted to get it working without using a declarative data source. This turns out to require a little bit of extra work. I found this article helpful in getting the DataPager working correctly. And this post had the best advice on how to control the visibility of the DataPager.

I think I could probably write up a couple of more posts with hints on the AsyncFileUpload control, but I’ll save those for another day.

Drupal 7

After staying out too late Tuesday night, then going to bed at 8:30pm last night, I finally had some free time after work tonight to play around with Drupal. I installed Drupal 7 on my MacBook, following these instructions. The only real trouble I had was in making sure that every host reference was set to 127.0.0.1 rather than localhost, or anything else.

Most of the obvious stuff seems to be working.  I haven’t figured out clean URLs yet though.

xAMP on the Mac

I’ve been trying to learn a bit about Drupal recently. It looks like we might be getting some Drupal projects at work, so it seemed like a good idea.

To get a working setup for Drupal on my Mac, I wanted to get all the pieces in place — Apache, PHP, and MySQL, basically. Apache is, of course, already there, and I already had that turned on, so no problem.

PHP was already installed, but apparently got turned off during the 10.7 upgrade. All you need to do to turn it on is edit httpd.conf, and uncomment one line, per this SO page.

For MySQL, there’s an installer that works pretty smoothly, per this page. One odd thing I stumbled across at one point is that you usually need to refer to your local server as ‘127.0.0.1’ rather than ‘localhost’. Long story, but something worth noting. Also, if you’re not sure how to set the root password, take a look at this SO page.

I tested to make sure that MySQL was working from PHP using this little test script:

<?php

$db = mysql_connect("127.0.0.1:3306", "root", "password");

if (!$db) {
    die('Could not connect' . mysql_error());
}
    echo 'Connected successfully';
?>

I can’t remember exactly where I found that, but it’s a pretty basic script.

I then got a little ambitious and decided to try to get phpMyAdmin working. I made a couple of simple mistakes here, including not quite understanding that config.inc.php needed to be in the root phpMyAdmin folder and not in the config subfolder.

Also, the warning from phpMyAdmin about mcrypt not being installed was bugging me, so I decided to try and fix that. That turned out to be kind of complicated. I followed these instructions, and they worked, but only on my second try. I must have gotten something wrong on the first try. Also, I found another page with similar instructions, so referencing that may help if anything on the first page seems confusing.

In the end, I think I really should have just gone with MAMP, but of course I was doing this as a learning exercise, so it was valuable to go through all this, even if it took a lot longer than was probably necessary.

And I still don’t have Drupal installed. Maybe tomorrow!