Category: programming

a little PowerShell

It’s been a while since I’ve posted any PowerShell code. I had to write a quick script today to run some SQL, save the output to CSV, then ZIP the CSV file. And I had to loop through and the run SQL multiple times, one for each month from January 2021 until today.

That forced me to look up some stuff that I didn’t know how to do in PowerShell, off the top of my head. (In fact, most of it was stuff I didn’t know off the top of my head. I don’t use PowerShell enough to remember anything…) So here’s an edited version of the script, simplified somewhat. It might come in handy some time, if I ever need to do this again.

# CustInvAll-export.ps1

#Requires -Version 7
#Requires -Modules SqlServer

$dateFmt = 'yyyy-MM-dd'
$sqlServer = "MyServer"
$dbName = "myDB"

$curDate = [DateTime]::Today
$startDate = [DateTime]'01/01/2021'
while ($startDate -lt $curDate) {
    $endDate = $startDate.AddMonths(1)
    $startDateFmt = $startDate.ToString($dateFmt) 
    $endDateFmt = $endDate.ToString($dateFmt)
    
    $exportSQL = @"
    SELECT *
    FROM MyTable
    where [INVOICEDATE] >= '$startDateFmt' and [INVOICEDATE] < '$endDateFmt'
"@
    $exportFile = "CustInvAll-$startDateFmt.csv"
    $exportFileZip = "CustInvAll-$startDateFmt-csv.zip"

    echo "Exporting from $startDateFmt to $EndDateFmt to file $exportFile"

    # Invoke-Sqlcmd -ServerInstance $sqlServer -Database $dbName -Query $exportSQL `
    # | Export-CSV -Path $exportFile -NoTypeInformation  -UseQuotes AsNeeded

    # Compress-Archive -LiteralPath $exportFile -DestinationPath $exportFileZip

    $startDate = $endDate
}

It can also be found in a Gist.

Programming Potpurri

I’ve been meaning to write a blog post or three about some of the programming-related stuff that I’ve been doing at work recently. But I keep putting it off. I’ve got some energy today, and a little spare time, so I’m going to try to write up some random notes.

Razor Pages

A while back, we had an old SharePoint 2013 page stop working. The page uses a control that I wrote in C#. The control really has nothing to do with SharePoint; it’s basically an old-fashioned ASP.NET web form that makes some web service calls, populates some controls, gathers user input, then makes another web service call, then redirects the user to another page. The only reason it’s in SharePoint is… well, that’s complicated. Let’s not get into that!

Anyway, fixing the page would take about five minutes. I’m pretty sure all I needed to do was increase a timeout, and increase the max receive size on a certain web service call. But… my SharePoint development VM got nuked in our security incident back in July. So the actual time to fix the error would be more like several days, since, at this point, I have no clue how to build a SharePoint 2013 development machine from scratch. I’m pretty sure I could do it, but it would take a lot of time and effort.

So I decided to just rebuild the page as a single-page ASP.NET Razor Page project, which seemed like it would be a fun thing to do, and might be a good model for moving some other stuff out of SharePoint. At the time, I wasn’t too busy. Of course, that changed, and now I kind of regret diving into this. But I did, and managed to learn enough about Razor to get the page done and into production.

I’d known a bit about Razor already, and had messed around with it on and off over the last few years. But most of my recent ASP.NET work has been web services, so there’s no need for Razor there. First, I was surprised to realize that Razor has been around since 2010. Scott Guthrie’s blog post announcing it is from July 3, 2010. I’ve still been thinking about it as “new,” but I guess it’s not. Heck, I guess it could even be considered “legacy” by some folks. (I guess maybe Blazor is what the cool kids are using now?)

Since it’s been around awhile, there are some reasonably good resources out there for learning it. But, also since it’s been around awhile, a lot of it is scattershot, or out of date, or not really relevant to what I was doing. The best resource I found is the Learn Razor Pages site. I almost bought the related book, ASP.NET Core Razor Pages in Action, but before I got around to it, I was pretty much done with the project, and had to move on to other stuff.

Dynamics 365

So, with the changes that are going on at work, it looks like I’ll have to be doing a lot more work with Dynamics 365. D365 is a pretty big topic. It looks like I’ll probably be mostly concerned with Dynamics 365 Sales (formerly known as CRM). I took a three-day class on Power Platform back in 2020, which is kind of the underlying technology for D365. Power Apps and Dataverse in particular are important. (The terminology on this stuff is really annoying. When I took that class two years ago, Dataverse was called “Common Data Service” and some of the other related terminology was different. It’s hard to keep up…)

I now have Pluralsight and LinkedIn Learning access via work, so I watched some videos on those sites, and on Microsoft’s Learn site, to refresh my memory from previous efforts to learn this stuff, and pick up on the new stuff. I guess I’m now almost at the point where I could be useful…

VSTO and EWS

Related to all that, I’ve been assigned to work on an Outlook plugin that ties into D365, and a console app that does some back-end processing related to the plug-in. So now I also need to learn VSTO, which is how the add-in was built, and EWS, which is used in the console app.

VSTO is a bit out of date, but not yet deprecated. If I was going to do a major rewrite on the plug-in, I’d probably switch to Office Add-Ins, which is a bit more modern, I guess.

And EWS is also out of date but not yet deprecated. If I wanted, I could probably move from that to the Graph API.

The main thing I need to do with these projects is to get them to work with Exchange Online. (We’re in the middle of migrating from on-prem right now.) I think I won’t actually have to change the plug-in at all, since it’s working with the Outlook object model, and I don’t think that cares if the email came from Exchange Online or on-prem. There might be a “gotcha” or two in there, though, so I need to at least test it.

For the console app, EWS still works with Exchange Online, but I know I’ll have to change a few things there, including switching over to OAuth for authentication.

And both apps seem to need some cleanup in terms of logging and error-checking. I know that if I make changes to these apps, then people are going to start coming to me with support questions, so I’ll need to make sure I have enough logging to provide support.

There’s actually been a lot of overhead involved in getting up and running on this project. These programs were originally under a different dev group, part of which has gotten moved into my group, so they’re using some conventions and utilities and stuff that I don’t know, and need to learn (and in some cases, gain access to). And I don’t have Outlook on my dev VM, since that’s not normally allowed (for security reasons). And I can’t get to the Exchange Online version of EWS, since that’s blocked (for security reasons). And I need to set up a new app registration, so I can access EWS with OAuth, and that needs to be approved by a global admin. And so on.

Was there a point to this?

If there’s a point to all this, I guess it’s just that I need to keep learning new things and being flexible. I saw a funny comic strip recently about an old man whose doctor tells him that he can help keep his memory sharp by learning new skills. And the old man says that his memory isn’t good enough for him to learn new skills. And of course I can’t remember where I saw that strip now, so I can’t link to it here. It was probably on GoComics, which I recently re-subscribed to, after canceling my subscription almost a decade ago. I’ve decided that reading the comic strips every morning is healthier than browsing Facebook and Twitter, so that’s why I re-subscribed. (I may also sign up for Comics Kingdom too, but that’s a subject for a different blog post.) Anyway, since I can’t find the strip I was looking for, here’s a different one, along similar lines.

A Busy Day

As previously mentioned, I got my COVID booster shot today. I also picked up my new iPhone 14 from CVS, though I had to make a separate trip for that, since it hadn’t arrived yet when I went for the booster. I guess the phone had gone back to the depot on Friday, then sat there all weekend, then got put on a truck this morning for delivery to CVS. Sigh. Well, I’ve got it now, so all is well.

This booster is my fifth shot, overall. The last shot was in April. I’m hoping I don’t get any side-effects from this one, since I’ve got a pretty busy week at work, and I don’t want to have to take a sick day. (But I will if I need to.)

The iPhone setup was relatively easy. I think the only major thing I still need to do is the MS Authenticator setup, which I’ve previously complained about. I’ll try to do that in the office tomorrow, where I’ll have access to my work desktop PC and my office phone, just in case any of my accounts still have the office phone number as a backup. I’m anticipating that’ll take about an hour.

Meanwhile, I had also ordered a new router this weekend, and that showed up today. I ordered this one, from Amazon. It was $80. I didn’t do a ton of research, bit it seems to be a successor to one that had been recommended by Wirecutter. I also found a good CNET review of it. I don’t have much to say about it yet, since it’s still in the box, and will likely remain there until this weekend.

I bought my current router in 2017, so it was time for a new one. The old one still works, but it doesn’t support some of the more modern features and standards. Honestly, I haven’t kept up with all that stuff, so I couldn’t even tell you which ones are which, at this point, but I know I was time for a new router. So I’ll try to get that set up over the weekend, probably, and I’ll have more to say about it then.

On another subject, there are a lot of changes going on at my job right now, and it looks like one of them may snowball into a pretty big change for me. (Or not. Hard to tell.) Either way, I think I need to learn a lot more about Dynamics 365 and Power Platform and stuff like that. I’ve made some efforts at learning that stuff in the past, but I never wind up actually working on anything real with it, so the knowledge doesn’t stick. And of course everything changes, so the stuff I picked up two or three years ago is different now anyway.

A lot of the changes we made in April are getting, well, changed. I wouldn’t say “rolled back,” but I am back under the boss I had before the big changes in April. But she’s under a different boss. And we seem to be backing off on our enthusiasm for scrum a bit. And, as mentioned above, I guess I’m going to need to learn more about the Dynamics 365 side of things. Which is good, I guess, but right now, my head is spinning.

Visual Studio extensions and alternatives

Another (relatively) quick post for today: It just occurred to me that, since my work VM has been restored from an older backup, the changes I made to Visual Studio a week ago are all gone. And that’s fine. I wasn’t entirely convinced that CodeRush was something I wanted to stick with anyway.

Since my VM is a little messed up now, I’ve been thrashing around a bit the last few days trying alternate ways to test some stuff out. I have some PowerShell scripts that I usually run on my VM, and I thought it would be fairly easy to copy them over to my laptop or desktop and run them there. But the specific set of modules I need is apparently a little complicated, and I haven’t been able to exactly recreate my environment. I’m sure I could, given time, but oh well.

Then I came up with the idea of installing LINQPad on my desktop and running some stuff there. That wasn’t terribly hard to do, but I’m referencing some DLLs in those that aren’t on my desktop, and, again, recreating the specific environment I needed got a bit too complicated and I gave up. That did get me thinking about upgrading my Pro license to a Developer license, so I’d have nuget support. I might still do that, but not right away.

And, finally, I have been looking at JetBrains Rider a bit lately. I noticed last week that their prices have gone up. Rider is still pretty affordable though. The old price was $139 for the first year (for personal use); the new price is $149. As a subscription product, that’s still enough that it’s not an impulse buy for me. I’d have to know that I was actually going to use it consistently, outside work. But lately I’m not doing enough programming outside work to justify it.

troubleshooting MSAL

This is just a quick one: in the aftermath of last weekend’s thing at work, they’re now blocking all outbound web traffic from my development VM. Which is kind of a problem when you’re trying to test out some web services on some restored servers, and you need to authenticate with MSAL. While trying to figure out exactly which addresses I needed to have whitelisted, I stumbled across an article on logging errors in MSAL.NET. This is a good example of something I needed, but didn’t know I needed until I stumbled across it, while looking for something else!

For now, I’ve just implemented this in my test program with the simple logging code included in the code sample, but, when things settle down, I might try to add MSAL logging into one or more of my programs as an option that I can turn on or off in the config, and log through Serilog (my current favorite logging library).

The end of a weird week

As I alluded to on Tuesday, we had a security incident at work over the July Fourth weekend, so we’ve been spending all week cleaning up after that. There’s a statement about it on my company’s blog. I won’t link to it here, since I don’t necessarily want this post to show up in searches or referral logs related to it. (I’m not going to post anything I shouldn’t be posting, and I don’t know enough about the details to say much anyway, but just in case…)

On Tuesday, we were told to just stay home and chill, basically. They didn’t give us any real info on what happened, so that was basically just a “snow day,” as I mentioned in my last post. On Wednesday, they had made enough progress with the initial mitigation that we could start working on our disaster recovery. So, since then, it’s been a lot of “hurry up and wait” work, where servers are getting restored or rebuilt, scanned, released, and then we can do final setup and testing. I’ve been in the office Wednesday and Thursday, but I’m hoping I can get away with working from home today.

There are a lot of people working very hard on this. For me, I’m primarily a programmer, with only limited admin responsibilities. There are only two servers that I’m “officially” responsible for (out of hundreds total across the company), so mostly I’m just waiting on other stuff, answering questions, and helping out where I can. I feel a little guilty, knowing that some people are sleeping in their offices while I’m going home and sleeping in my own bed, but of course there’s not much I can do to help those folks, other than to stay out of their way.

I had a bunch of other stuff to say about this, but I think I’ll just say that I think we’re doing a pretty good job of handling this thing. Everyone has been calm and professional through it all, at least from my limited vantage point. There’s still a lot of work to do, but we’re making steady progress.

Purely by coincidence, I read an article in Communications of the ACM last week that talked about the best way to handle outages in IT. The article is more about unintentional errors that cause outages, rather than security incidents, but there’s a lot in common. I like the idea that “DevOps celebrates mistakes.” I hope that, when we’re back up & running, we’ll get a good postmortem on this that we can learn from.

Visual Studio extensions and tweaks

I’ve been spending some time at work recently messing around with my Visual Studio setup. I’ve been fine with my current setup for awhile, but I started getting restless recently. I guess it started when I started reading Clean Code, and watching the associated videos. That got me thinking about automated refactorings, which got me looking at JetBrains Rider and Resharper. And looking at Resharper reminded me of the existence of CodeRush.

I’ve occasionally thought about trying out something like Resharper or CodeRush, but I never got around to it. There are a number of reasons for that, mostly around the cost and the possible performance penalty. But I noticed recently that CodeRush is now available for free, so I figured I’d give it a shot. (And I think Roslyn made it easier for extensions like CodeRush to work without a big performance penalty.)

My normal VS setup, which I’ve stuck with for a while now, is pretty basic, with Mads Kristensen’s Web Essentials, and DPack Rx. I use DPack primarily for the numbered bookmarks, and Web Essentials for a number of random things. CodeRush includes numbered bookmarks, so I thought I’d try removing DPack, installing CodeRush, and seeing how that worked out. So far, I’ve found that CodeRush’s numbered bookmarks don’t work quite as well as DPack’s. There are a number of other interesting features in CodeRush, but I’m not sure if any of them are compelling enough for me to keep CodeRush installed.

I also briefly considered uninstalling Web Essentials, and then just reinstalling the specific extensions from that collection that I’m actually using. But I couldn’t quite talk myself into that, so I’ve still got the whole collection installed.

On a related subject, I recently listened to an episode of .NET Rocks with Mads Kristensen on VS 2022 extensions. And another one with Mark Miller (author of CodeRush). I’m not currently subscribed to .NET Rocks in my podcast client, but I do check in on it occasionally to see if there’s anything interesting.

Getting back to the general subject of VS extensions and setup, I also revisited my work setup and tried to decide if I could switch from VS 2019 to VS 2022. I do my VS development at work on a VM that’s running Windows Server 2012 R2. That probably seems weird, but it’s necessary for Dynamics AX development. And VS 2022 isn’t supported on Server 2012, though I could probably get it to work. So I’ve been going back and forth on whether or not I should try to install it on my current VM, or maybe ask my boss for a new Windows 10 VM to use for VS 2022. I’ve decided to stick with VS 2019 for now, but I may need to ask for a new VM at some point. In part because I have a .NET 5 project in VS 2019, and .NET 5 is no longer supported. And .NET 6 isn’t supported in VS 2019. So it’s all kind of complicated. Asking for a new VM shouldn’t be a big deal, but I’m a little nervous about it, since I just got a new boss, and I’m now in a different sub-division of the IT department, and the rules about this stuff might be a little different than they were under the old boss, so I want to feel things out a bit before I start asking for stuff.

I did install VS 2022 on my personal laptop back in May. I do like it, and would love to be able to switch over to it. (Unlike some previous versions of VS, they don’t seem to have made any really bad UI decisions that make me want to stick with the older version…)

I also spent a little time messing with the default font in VS on Friday. I’ve stuck with Consolas for quite some time now. But  I was watching a LinkedIn Learning video on Friday where the teacher’s setup was using Cascadia Code, and it looked kind of nice. I’d read about Cascadia Code when it came out, but I never got around to trying it. So I switched over to it for a little while, but then decided to switch back to Consolas. The whole code ligature thing is interesting, but Consolas just seems to work better for me.

All of this fiddling around made me think about the balance between sharpening the saw and… pointless procrastination. (It’s bothering me that I know that there’s a clever metaphor similar to “sharpening the saw” that basically means “pointless procrastination”, but I can’t remember what it is.)

Well, I’ve now killed a bunch of time on a hot Saturday morning, drinking iced coffee, waiting on my grocery delivery, and writing a rambling blog post. My plan for the rest of the day revolves mostly around watching the last two episodes of Stranger Things on Netflix. Life is good, I guess.

Lazy initialization

My experience working my way through the Clean Code book and videos has gotten me interested in refactoring some of my old code. I have a large API project that I maintain at work, and it’s gotten a little out of hand over the last few years. In particular, there’s one class that got so large that I broke it up into multiple partial class files some time ago. Overall, it would have been around 5000 lines of code if I hadn’t broken it up. Of course, partial class files don’t really change anything; it’s still one big class, technically.

So I started looking for ways to break it down into actual separate classes. What I wound up doing was to leave stuff that was needed across all the code in the old class, then moving all the specific stuff from the partial classes into new “child” classes. I created a new base class for these child classes, then added code to the original (now “parent”) class to contain all of the children as singletons, instantiated as needed.

The code to do that looked a lot like the code in this Stack Overflow question. That was fine, but once I got all that done and working, I started looking around for ways to streamline or simplify the declarations. I tried out a few things, and decided on using the Lazy<T> class. I’m sure I must have stumbled across it before, but I’d forgotten about it and had never actually used it. It took me a little while to figure out exactly how I should use it, in my particular use case. I don’t have anything super-interesting to say about that, but I thought I’d just post a few links that I found helpful:

Overall, I think the code is a little better for having been refactored this way, though I’m still not quite satisfied that I’ve cleaned things up as much as I should.

Comparing objects in .NET and general thoughts on testing

I had a little task today at work, where I needed to replace the way I created a JSON object that was being returned from an API. Long story, but the challenge at the end was to make sure I didn’t screw it up, so I wanted an easy way to compare the “before” and “after” JSON. Now, I was adding a feature here, so the before and after were going to be different, but I wanted to have an easy way to check that they were different only where I expected them to be, and there weren’t any unexpected side-effects.

This let me down a rabbit hole, looking at things like this ObjectsComparer package on GitHub. I also stumbled across this JSON extension for FluentAssertions. In the end, I just wrote a script in LINQPad to dump the before & after JSON to disk, then compared it with Beyond Compare. I keep thinking that I need to put together a better testing framework for my API, so I can better automate this kind of stuff.

I don’t have a unit test project for this API, or any other really structured tests. I have a test client that can run a suite of “safe” tests that don’t alter data, and checks a few things on the results. But it’s not a very complete test set. And I have another client that runs a test cycle that does alter data, and hence needs to be run carefully, and can’t be run too frequently. (And that one isn’t very complete or thorough either.) And I have some LINQPad tests that I run mostly as smoke tests.

I’m currently working my way through the TDD section of the Clean Code series that I’ve been watching/reading. I’ve always liked the idea of TDD, but I’ve rarely worked on a system where it seemed practical. (I’m not saying that it’s not practical, just that I haven’t figured out how to effectively apply it to any of my typical work.) Regardless of TDD, I ought to be able to put together some better and more automated tests for some of my work.

Clean Code, Visual Studio, Windows 11, and a bit more on ebooks

I wanted to add some more notes about the whole Kindle, EPUB, MOBI, etc. thing that I’ve been blogging about in my last few posts. As I mentioned a few posts back, I’ve been working my way through the Clean Code learning path in O’Reilly. Since I’m probably going to lose access to O’Reilly before I’m done with that, I thought I’d buy a copy of the Clean Code book, so I could finish reading it at my leisure, and to have it for later reference. With tech books, I usually like buying a DRM-free ebook directly from the publisher, when possible. In this case, I initially had some trouble finding that, but eventually figured out that it’s purchasable through the InformIT site. I bought the Robert C. Martin Clean Code Collection ebook, which contains both Clean Code and The Clean Coder. I had a discount code, so it was about $40 total.

I copied the EPUB, MOBI, and PDF files for the book to OneDrive. There are a number of ways I can read an ebook on my iPad, if it’s DRM-free and available in multiple formats. For this one, I wound up sending it from OneDrive into my Kindle app, as a MOBI file. That method still doesn’t support EPUBs. And it will copy the file up into my Kindle library, which is nice. I don’t think I’d ever copied anything into the Kindle library that way before, but it worked fine. I also copied the PDF into GoodReader. The code listings in the MOBI version look a little weird, which is a common problem with tech ebooks, so it’s nice to have the PDF handy as an alternative.

Clean Code has some example refactorings that are fairly interesting. They’re all in Java, but I thought it might be interesting to take the original Java code for one of the examples, convert it to C#, get it working, then work through some refactoring that’s similar to what Uncle Bob does in the book/videos. I found some of the code for the examples in GitHub under the Clean Code Kata user account. (I’m not sure if that’s an “official” account for Bob Martin or his organization, but either way, the code is there.)

This idea to convert some of the examples to C# and work through them was prompted in part by a desire to set up a .NET dev environment of some sort on one of my personal machines, and to maybe experiment a bit with some of the more recent .NET stuff, like .NET 6 and VS 2022. I realized that I don’t currently have any dev stuff at all set up on my desktop PC, my MacBook, or my Lenovo laptop. The MacBook is new, so I just haven’t set any dev stuff up yet. The Lenovo was bought in 2020, and I haven’t gotten much use out of it at all. And I’ve been trying to keep the desktop PC free of any heavyweight dev tools, since I just want to keep it clean for personal productivity stuff. After going back and forth on a few possible setups, I decided to install Visual Studio 2022 (Community edition) on my Lenovo laptop. I considered just installing the .NET 6 SDK and Visual Studio Code, which would have been much more lightweight, but I’m used to using the full VS product, and I can’t see a reason not to use it. And the Lenovo is the best place to install it, since I can wipe out that machine entirely and start fresh if things get too messed up. I’ve also recently upgraded that laptop to Windows 11, so this was also an opportunity to (finally) give that a try.

Of course, I’ve had other things to do this weekend too, so I’ve only gotten as far as installing Visual Studio and git, and tweaking some settings. Maybe I’ll actually do some programming next weekend. (Or maybe I’ll get distracted by something else, and the whole thing will fall by the wayside.)