Stuck In The Mud With SPFx

I’ve been trying to make some progress with SharePoint Framework (SPFx) lately, but I keep getting stuck in the mud, so to speak. I started working on learning SPFx some time ago, but I had to put it aside due to other projects. But now, I have a little spare time to get back to it.

I set aside a few hours one day last week to work on it. But since I last worked on it, I’ve moved most of my work to a new dev VM. So step one was moving all of my SPFx projects over to the new VM. That shouldn’t have been a big deal. But of course each SPFx project has a node_modules folder of about 725 MB, across more than 100,000 files. So just copying everything over wasn’t going to work. So step 0.1 (let’s say) would be to delete the node_modules folders. Since I had less than a dozen work projects, I thought I’d use brute force for that, and just click each node_modules folder in Explorer and hit the delete key on my keyboard. Of course I then realized that asking Windows Explorer to move 100,000+ files to the recycle bin is a bad idea. So I started looking into writing a script to do it.

I found something called npkill that looked like it would do the trick without me even having to write a script, but I couldn’t get it working in Windows. (It’s probably possible to get it working in Windows, but I hit a snag and decided not to spend too much time on it.)

So I was back to writing a script. I started putting something together in PowerShell, but then I found rimraf, which looked promising and (according to at least one blog post I read) would be faster than doing the equivalent recursive delete natively in PowerShell. So I wrote a PowerShell script using rimraf. I wound up with this simple one-liner:

gci -name | % { echo "cleaning $_\node_modules..."; rimraf $_\node_modules }

I’m not sure if rimraf was actually faster than just using a native PowerShell command, but it worked. So that got me down to a manageable set of files that I could zip up and move to the new VM. (There was actually some trouble with that too, but I won’t get into that.) And that pretty much killed the time I’d put aside to work on SPFx for day one. Sigh.

For day two, I wanted to get back to a simple project that would just call a web service and return the result. I’d previously stubbed out the project with the Yeoman generator on my old VM, so now I just had to do “npm install” to get the node_modules folder back. Long story short, I got some unexpected errors on that which led me down some rabbit holes, chasing after some missing dependencies. That got me messing around with using yarn instead of npm, which someone had recommended to me. That didn’t really help, but after a bunch of messing around, I think I figured out that the missing dependencies weren’t really a problem. So just messing around with npm and yarn, and getting the project into a git repo, killed the time I’d set aside on day two.

For day three, I actually went into the project and added a web service call, to a local service I wrote, but immediately hit an error with the SPFx HttpClient not liking the SSL certificate on that web service. So that got me trying to figure out if you can bypass SSL certificate checking in the JavaScript HttpClient the same way you can in the .NET HttpClient. I got nowhere with that, but it did set me down the path of looking into that SSL cert, and realizing that it’s due to expire in January, but I didn’t have a reminder to renew it in Outlook. Which got me going through all of my SSL certs and Outlook reminders and trying to make sure I had everything covered for anything that might expire soon. And that sent me down a couple of other administrative side-paths that used up all the time I’d set aside on day three.

So after three days, I basically just had a sample SPFx project that makes one simple web service call, which fails. Sigh. I picked it back up today, trying to fix the call. I got past the SSL issue. But that led me down a couple of more rabbit holes, mostly regarding CORS. So, good news: I now understand CORS a lot better than I did this morning. Bad news: I spent most of the morning on this and can’t really spend most of the afternoon on it.

At some point, I’ll get over all these initial speed bumps and actually start doing productive work with SPFx. Maybe.

SharePoint, React, Laptops, and so on

I mentioned a while back that I’m trying to learn about the (relatively) new SharePoint Framework (SPFx), for a project at work. I’ve made some progress with that, but I still have a way to go. I’ve done 5 of the 8 modules in this course from Microsoft. And I’ve watched a couple of Pluralsight videos, one from Sahil Malik and one from Danny Jessee. I’ve been doing that mostly on work time, since it’s specific to a work project.

SPFx relies on a number of related technologies, some of which I know and some of which I don’t. (And the ones I know, I don’t necessarily know that well.) So I decided to start digging into some related stuff, on my own time. I know pretty much nothing about React, and it looked interesting, so I decided to start learning that. I’ve watched one short Pluralsight video, that just gives an overview without getting into specifics. And now I’m working through a four-hour video course that goes into a little more detail. There’s a whole skill path for React on Pluralsight that would take about 40 hours to watch, if you went through it all. (And of course it would be much longer than that, if you actually followed along and worked through projects on your own.)

I got side-tracked off of React at one point when I was watching one of the Pluralsight videos on my old ThinkPad, and the battery suddenly died. I’ve had that laptop since 2011, and it’s starting to show its age. I’d only been watching the video for about 30 minutes, and the battery should have had a full charge when I started. So I started thinking about either replacing the battery on it, or just getting a new laptop. Replacing the battery on that particular model is really easy. And there were a bunch of options for a replacement battery on Amazon (though most of them looked kind of sketchy). But I started thinking about how old the laptop was, and how iffy off-brand replacement batteries can be. And I also started wondering if that laptop was going to be able to handle some the stuff I’m going to want to try out soon, like WSL 2. I’ve been hearing about that for a while, and it’s now been released as part of the Windows 10 2004 update. The old ThinkPad, surprisingly, has been able to keep current with Windows 10 updates so far, up to version 1909. But I have my doubts about whether or not it’s going to be able to deal with 2004. So, reluctantly, I started shopping for a new laptop.

This is a pretty common thing with me: I start trying to learn a new technology, and I get side-tracked shopping for a new laptop, or some new piece of software, or something. Anyway, I spent way too much time on that yesterday. This morning, I finally settled on the Lenovo Flex from Costco, for $750. It’s a bit of a compromise, since I’ll need to upgrade it to Windows 10 Pro, but I can still do that for $40 with my Microsoft company store access, which should still be good for the next week or two. Also, it’s a 2-in-1, which I don’t really need or want, but most Windows laptops seem to be touchscreen 2-in-1 models now, so I’ll give it a try. On the positive side, it’s got 16 GB of RAM, a 512 GB SSD, and an AMD Ryzen 7 CPU. (I haven’t really been keeping up with CPU news lately, but it looks like the AMD Ryzen 7 4700U is pretty good.) So I think it should be able to handle my fairly modest needs. I always feel a little guilty when I spend money on new hardware, but I’m trying to remember that, this year, I’ve spent nothing at all on travel, and I’m not likely to. If I’d gone to WonderCon this year, that would have cost me well over $1000, for hotel and airfare alone.

I was going to remark that I’d made it through a whole post without referencing COVID-19, but the travel comment above kind of does reference our current situation, so I guess that’s not true. COVID-19 definitely did affect my laptop shopping. In normal times, I probably would have gone out to Costco yesterday to see what laptop models they had on display. And I might have taken a trip to Best Buy too. Costco is still open, but I don’t really want to go there unless I have to. And Best Buy of course is still closed. So I settled on a mail-order laptop from Costco. They have a good return policy, if I need it.

SharePoint, Somerville, and so on

A little follow-up on some subjects from yesterday’s post:

I complained a bit yesterday about the “hundreds of files” pulled in on a new “Hello World” SharePoint Framework project. I checked today, and it’s actually more than 50,000 files, totaling up to about 500 MB. Scary. I’ve also been a little worried about all the security warnings issued by npm when scaffolding a SPFx project. Apparently that’s all fine though and I should just ignore them, according to this blog post. I guess none of the stuff that npm is checking is actually ever deployed to SharePoint, so it’s fine.

NJTV News tonight had a segment on restaurant and retail reopenings that spent some time talking about Somerville. I guess we’re likely to go ahead with the plan to close down Main Street to car traffic a few nights a week that I mentioned yesterday. I’ve still got some reservations about that, but nobody asked my opinion. (Yeah, I know, I could start attending town meetings. They’re virtual now, so I don’t even need to leave my couch. I’m still probably not going to do it though.)

One other benefit of having “attended” Microsoft Build this year: They’re letting attendees buy some stuff from the Microsoft company store. They’re only allowing purchases of digital goods, so no discounts on Surface hardware or anything like that. But I did pick up a few things at bargain prices. I got a Windows 10 Pro license for $40, and used it to upgrade my desktop PC from Home to Pro. And I got a one-year extension on my Microsoft 365 Family account for only $20. (That’s usually $100/year. I get the Home Use Program discount, which makes it $70/year. So $20 is really low.) And I got a two-year Xbox Live Gold sub for $50. (That’s usually $10/month or $60/year.)

I don’t know if I’ll actually get much use out of the Xbox Live Gold account. As I mentioned recently, I’ve had the Xbox for a year now, and I barely use it, except as a DVD/Blu-ray player. I’ll have to keep an eye on the Games with Gold stuff and see if they have anything I’m interested in. I really want to start playing video games again, but there’s so much other stuff to do too.

SharePoint, social distancing, civil unrest, and so on

I need to start a new SharePoint Online project at work soon. It’ll be an attempt to move an on-prem SharePoint 2013 site, with a fair amount of custom code, to SPO. I haven’t had time to learn much about SPO yet. I’ve taken a couple of pokes at it, but I’d been having trouble finding the right resources.

I “attended” Microsoft’s virtual Build conference this year, and had hoped for some useful SharePoint content, but there wasn’t much. About the only thing I could find was this session on the Microsoft 365 developer program. I already knew about that, and have an account, so that wasn’t too useful. It did, however, point me in the direction of a web page that (in turn) pointed me to this course on extending SharePoint. That seems to be what I need to get started.

I’m cautiously enthusiastic about learning this stuff, but I’m a little leery of the dev stack that they’re recommending. I have some limited experience with the tools they’re using (gulp, yeoman, node.js, and so on), but this stuff always seems like a house of cards to me. Too many different tools, all from different open source projects, pulling in possibly hundreds of different files, all just to get the scaffolding for a “Hello World” project up and running. Well, I need to remain positive and give it a try. I made it through the first “Hello World” example today, and I’m hoping I’ll have time to make some more progress tomorrow.

Since the dev stack includes node.js, I found myself visiting the node.js web site today. They’ve changed their home page to contain a Black Lives Matter message. (I’m not sure how long they’ll leave it up, so here’s a link to an archive.org snapshot.) We had a fairly small and very peaceful BLM march in Somerville over the weekend. And protests in NJ have mostly been peaceful, with some exceptions. I don’t have much to say about all this, other than that I hope something positive comes out of it all. I’m afraid that it’s going to get worse before it gets better though. (My own contribution to this situation was to start catching up on all the Black Lightning episodes on my TiVo. And to keep listening to the Invisible Man audiobook that I started a while back. So, not much, really.)

Meanwhile, NJ is starting to open back up a bit. Today actually marks three months since the first COVID-19 case in NJ, according to the newscast I just watched. I think that Murphy is acting with a reasonable level of caution, all things considered. I am worried about the “knuckleheads” who might push things a little too far and cause another spike in cases. I’ve actually been venturing out a bit more myself this week. I had a doctor’s appointment, then had to go to Quest for some blood work. And I’ve got a dentist’s appointment next week. It feels a little weird, going out and driving and stuff. I’m really wondering about how “armored up” the dentist and hygienist are going to be for my appointment. Dental work has got to  be pretty high-risk, given the level of contact necessary.

Here’s an article about the current state of things in downtown Somerville. And here’s one on a plan to close off Main Street to car traffic a few nights a week, and use the road for outdoor dining. It’s an interesting plan, though if it’s not implemented carefully, it could be a disaster. I want to see Somerville’s restaurants have a chance to do some business this summer, but not if it means that the whole street is crammed with people eating and drinking and spreading germs. If they can keep things reasonable and organized, maybe it’s not a bad idea. If things get crowded (like on a normal, pre-COVID-19, Friday night), then I’m going to be locking myself in my apartment and keeping the windows closed.

more TypeScript and SharePoint

I got a chance to work on another TypeScript / SharePoint project at work recently. I’m finding these projects to be a nice change of pace from my usual Dynamics AX work.

I’m not doing anything that’s much more complicated than I did on the last one, but I got a little more ambitious on this one. I’m still using Q for promises and Papa Parse for parsing an input CSV file. I think I’m getting a little better with promises, but I still occasionally have problems with them.

On my last project, I put all my TypeScript code in one file. For this one, I’ve broken things up into several classes and files. I’ve even gone as far as having an abstract base class with two child classes inheriting from it.

And, for the last project, I just deployed the generated JavaScript as-is. For this one, I’m using Web Essentials to combine and minify the JavaScript. That’s working better than I thought it would.

I’ve also given up on the idea of deploying this thing as a regular SharePoint sandbox project, though I’ve set it up that way in Visual Studio. Instead, I’m simply copying the HTML and JS files to the SharePoint server with SharePoint Designer. Maybe that’s not the best way to do it, but it’s the easiest, and the least disruptive to the server. (I should look into whether or not it’s possible for me to do that with PowerShell.)

programming books and videos

We’re snowed in today here in my part of NJ, so today’s probably a good day to review and clean up some of the “independent study” stuff I’ve been working on over the last several months.

First, I decided to finally finish up a book that I started reading about a year ago, Real-World Functional Programming. I’d been reading it a little bit at a time for quite a while. I started in on a program to learn F# back in 2014. I read a few books, and learned a bit, but I never really got a chance to apply any of that knowledge on a practical project. So I skimmed through the last couple of chapters of that book today, marked it as “read” in Goodreads, and decided that my F# experiment is over for now.

I started (and finished) reading a book on JavaScript this week, Object-Oriented JavaScript. I’m doing a bit of JavaScript programming at work right now, but I’m rusty, since I haven’t used it much lately. I got this book for free at some point from Packt, so I thought maybe it would be a good way to brush up and refresh my memory. It was a good refresher, and even had some stuff in it that I hadn’t stumbled across before.

On the video front, I’m still working my way through SharePoint videos on Pluralsight. I’ve completed Andrew Connell’s “SharePoint 2013 Developer Ramp-Up” series, and Sahil Malik’s “Understanding SharePoint 2013” series. Now I’m working on David Mann’s “Developing SharePoint 2013 Solutions with JavaScript,” which is helping me out with the SharePoint/JavaScript stuff that I’m currently working on. When I paid for a year’s worth of Pluralsight, I wasn’t sure if I’d get my money’s worth out of it, but I think I’ve been making good use of it so far this year.

I’ve also now been sidetracked into messing around with TypeScript. I read a book on CoffeeScript a few years back, but CoffeeScript never really took off (at least in the .NET community), while TypeScript seems to be very popular right now. (Take a look at this Google Trends graph.) So I’ve been experimenting with using TypeScript and JSOM together in a SharePoint project. I’m not sure if it’s worth the effort, but it’s interesting. I haven’t devoted too much time to TypeScript yet, but I’ll probably watch a Pluralsight video or two on it and see if I can persuade myself into using it.

Finally, I feel like I should get back to Ruby on Rails at some point. I started learning Ruby back in 2015, and learned the basics (of both the Ruby language and the Rails framework) but really didn’t get as far as I wanted. I got partway through Michael Hartl’s book/tutorial, but I guess I got off track at some point, since I haven’t touched it since June 2015. As with F#, I never had any real project in mind, or work-related reason to learn Ruby, so I probably abandoned it in favor of something else I needed to learn.

So I guess I’ve got some goals for the rest of 2017: keep working on SharePoint, brush up on my JavaScript some more, look into TypeScript more deeply, and maybe get back to Ruby on Rails, if I have time.

Getting master page URL in SharePoint 2013 with JSOM

I’m only just starting to learn JSOM (after previously learning the server-side object model, and CSOM, via C#). There are some slightly weird requirements and limitations that are constantly tripping me up. Here’s today’s waste of time: trying to get the current site’s master page URL.

Here’s what I came up with:

$(document).ready(function () {
            $('#divMain').text('Hello World.');
            getMasterUrl();
        });
        function getMasterUrl() {
            var myCtx = new SP.ClientContext();
            this.site = myCtx.get_site();
            this.web = myCtx.get_web();
            myCtx.load(web);
            myCtx.executeQueryAsync(
                Function.createDelegate(this, this.onSucceededMasterUrl),
                Function.createDelegate(this, this.onFailed)
                );
        }

        function onSucceededMasterUrl() {
            $('#divMain').text('Master URL: ' + web.get_masterUrl());
        }
        function onFailed(sender, args) {
            $('#divMain').text('Failed: ' + args.get_message());
        }

This just kept failing. I tried a bunch of random stuff to get it to work. In the end, I made a fairly minor change that got it to work. But then, I changed the code back, and it still worked. So this was going to be a post about a minor quirk of JSOM, but now it’s more of a “WTF” post. I’m sure I did something to make it work, and I just lost track of what it was. Well, either way, I’m having some fun with JavaScript.

SharePoint: plain text, rich text, AutoHyperlinking, and Markdown

I’m just about finished with the SharePoint project that I’ve been working on for the last few months. One requirement for the project was to allow arbitrary “comments” on the main documents for the project. There are some built-in ways to accommodate comments in SharePoint, but I gave up on those after experimenting a bit. Instead, I created a new list that would act as a child table to my main list, in a simple one-to-many relationship. And I decided to use plain text (rather than rich text) for the comment field itself.

I’ve had problems with SharePoint rich text fields in the past, and I wanted to put some constraints on the users, so they wouldn’t go nuts with the vast array of bad things rich text fields in SharePoint let you do. And I didn’t see any reason why plain text wouldn’t be “good enough” for this particular case. However, for this application, a lot of URLs and email addresses are going to get posted in comments, and I wanted to be able to “linkify” them. I almost wrote my own code for that, but then found the SPUtility.AutoHyperlinking method. It works pretty well, and also translates quotes, angle brackets, and other possibly confusing characters into their corresponding HTML entity codes.

I also got a little interested in the idea of supporting some limited formatting (like bold, italic, etc.) without going full-on rich text. My first thought on that was to look into the SharePoint wiki functionality. I was hoping for a function like SPUtility.AutoHyperlinking, but which would convert some simple wiki markup into HTML. But SharePoint’s wiki capabilities are limited, and really only support links.

So I then gave Markdown some thought. There’s obviously no built-in support for Markdown in SharePoint, but I figured that I could find a .NET library that would let me handle the MD to HTML conversion on the back-end. There are, indeed, several libraries available for Markdown conversion. I found two that stood out as probably the best, for my use:

  • CommonMark.NET is a pretty popular one that’s been around for a while.
  • Markdig looks like it’s probably newer and slightly less popular than CommonMark.NET, but it has some interesting extensions, including an auto-linking extension that would have been useful for me.

In the end, I decided that it was pretty unlikely that the user base for this project would embrace anything as nerdy as Markdown, so I didn’t bother adding it to the project. But I had some fun messing around with it.

And I should mention that I figured out, at some point, that SharePoint 2013 supports two levels of rich-text: one that is the “full” rich text mode, allowing pretty much anything and everything, and one that is limited to a pretty reasonable subset (bold, italic, text alignment, links, and stuff like that). In retrospect, I probably should have gone with the limited rich-text, though even that might have caused unexpected issues. (I have learned to trust SharePoint only as far as I can throw the server on which it’s running…)

SharePoint – fun with fonts

I hit another weird little problem on my SharePoint project today. This one’s a bit different from the previous ones I’ve blogged about recently. A key point to start: I’m developing my solution on a VM running Windows Server 2012 R2. But the end-users are all using Windows 7.

I have one big detail page for this  project that’s got a lot of information on it. It’s a regular Web Forms ASP.NET page, in a SharePoint farm solution. I’ve tried to get everything on the page looking reasonably nice, while staying within the default look and feel of SharePoint 2013. So I’ve just got some CSS tweaks to get everything laid out right and looking good. And the page does look reasonably good on my dev VM. But I’ve noticed that certain text looks pretty bad when viewed on a Windows 7 machine.

The default font in SharePoint 2013, for most stuff, is something called “Segoe UI Light”. This is a Microsoft font that they, apparently, use for a lot of internal stuff. If you look at this page, you’ll see something interesting: Windows 7 uses version 5.00 of the font, while Windows 8 uses version 5.27. Checking my desktop Win 7 PC, I can see that it is indeed on version 5.00. (And I have version 5.36 on my Win 2012 R2 VM.)

This blog post goes into the differences between these font versions in a bit more detail. Here’s the one line that really caught my attention: “Microsoft’s fonts team has also worked on improving the hinting of Segoe UI, especially the Light variant which was never properly hinted.” So, yeah, that “never properly hinted” thing is probably why my page title looks horrible on Windows 7.

I don’t want it to sound like I’m bashing Microsoft’s font too much. It’s actually pretty nice, especially if you have a recent version on your PC and not the 5.00 version. But, for my project, it’s a problem. So I looked into switching to a Google web font. I choose Open Sans as a replacement for Segoe UI. I’d seen it suggested somewhere, and it seems to work well, and is free to use.

I’ve used Google fonts before, but had forgotten how to use them. It’s pretty easy. Just Put this in your page head:
<link href="https://fonts.googleapis.com/css?family=Open+Sans" rel="stylesheet">

And use this for your CSS:
font-family: 'Open Sans', sans-serif;

This has worked out pretty well for me. The page now looks good on Windows 7 and on more recent versions.

More SharePoint list view threshold fun

Here’s a quick follow-up to my previous post on dealing with SharePoint list view thresholds. I just bumped up against another case where I had to change some code to deal with it.

To recap the project a bit, I am writing a console app that will import a bunch of data into a SharePoint site. Since I’m human, and I make mistakes, the first step of this importer is to delete any existing data in those lists (which would be leftover from the previous test run).

I had a simple solution for doing that, deleting records in batches of 100, based somewhat on this example. I assumed that would work OK, even with larger lists, but I didn’t take into account that the first step in my process was to get all items in the list. That, of course, fails for a very large list. So I had to change my code to initially get only the first 4000 records in the list. (That can be done with the RowLimit clause in CAML.) Then, I delete from that subset in batches of 100. Then, I just repeat that until there are no more records left.

As far as I can tell, there’s no SharePoint CSOM equivalent to SQL’s “truncate table”, which would have made this much easier. And I feel like I’m probably still not doing this in the most efficient way. If I was creating a process that needed to do this repeatedly, instead of just a few times, I’d dig into it some more. And if I was retrieving items instead of deleting them, I’d probably do something with ListItemCollectionPosition.


private void deleteAllFromList(ClientContext cc, List myList)
{
int queryLimit = 4000;
int batchLimit = 100;
bool moreItems = true;
string viewXml = string.Format(@"
<View>
<Query><Where></Where></Query>
<ViewFields>
<FieldRef Name='ID' />
</ViewFields>
<RowLimit>{0}</RowLimit>
</View>", queryLimit);
var camlQuery = new CamlQuery();
camlQuery.ViewXml = viewXml;
while (moreItems)
{
ListItemCollection listItems = myList.GetItems(camlQuery); // CamlQuery.CreateAllItemsQuery());
cc.Load(listItems,
eachItem => eachItem.Include(
item => item,
item => item["ID"]));
cc.ExecuteQuery();
var totalListItems = listItems.Count;
if (totalListItems > 0)
{
Console.WriteLine("Deleting {0} items from {1}…", totalListItems, myList.Title);
for (var i = totalListItems 1; i > 1; i)
{
listItems[i].DeleteObject();
if (i % batchLimit == 0)
cc.ExecuteQuery();
}
cc.ExecuteQuery();
}
else
{
moreItems = false;
}
}
Console.WriteLine("Deletion complete.");
}