A few days ago I installed some malware by mistake, and decided it was time for reinstalling Windows. I’m a huge fan of fresh installs, and do it at least once or twice a year.

Since I use SugarSync online backup for all my important files, I like to heep them handy, all inside a single folder, out of Windows/Users folders (actually they are in a different partition, which helps me to format Operating System partitions without fear). Since I have all important files outside of their default locations, I always have to reconfigure the location for Documents, Pictures, and other folders. Also, since natively you can only change the location for some very specific folders, my fresh installs usually contain some junctions. And since in a few cases I also need to redirect on a file-basis, I also use hard links, which help me to keep safe some individual-files like Filezilla bookmarks, and hosts file.
(Click here to learn more about junctions, hardlinks, and mount points).

With that design, I never have to worry about formating the OS partition, and don’t have to worry about backing up multiple subfolders. It’s all backed up daily in real-time.

To keep that scenario I also have detailed step-by-step instructions on how to configure Windows the way I like it, which include:

  • Dual boot on SSD. One Windows for serious work, and the other for testing different software and installing things that I don’t use very frequently.
  • Ubuntu on another HD partition for open-source work.
  • Small TEMP partition on the start of a HD, for faster writes
  • Paging file on fast TEMP partition
  • TEMP/TMP environment variables pointing to TEMP partition

The instructions include some tips and reminders like:

  • Using YUMI to create bootable USB
  • Which Windows Updates should be skipped and which should be installed
  • Drivers downloaded for offline usage (specially network adapter, in case Windows does not install automatically)
  • How to configure SSD for optimal performance
  • Reconfigure Power Options, Regional Settings,
  • Configure /etc/hosts
  • Which programs to reinstall, in which order (e.g. SQL Server before Visual Studio), and some configurations.

During this last Windows Reinstall, I faced some problems with OneNote, which was not pasting images correctly. Calling onenote /safeboot and clearing cache and settings (which is a default solution for many problems) didn’t help at all. So I had to do the troubleshooting myself.

I fired up Process Monitor, which traces all read/write attempts both to filesystem and registry, and tried to paste into OneNote again. Then I reviewed the log, filtering only for Onenote.exe, and filtering out all “Success”. I was pretty sure that it was a problem in filesystem, and not in registry (because all my changes were related to filesystem, partitions, and folders), so I also filtered out everything related to registry. This is what was left:

I noticed that OneNote was trying to open C:\Windows\TEMP\Drizin, and it was being reparsed (by a junction) to T:\Drizin. My suspect was that the reparse point (junction) was the problem (and not permissions). So I replaced my environment variables (TEMP and TMP) to point directly to T:\%USERNAME%, instead of pointing to C:\Windows\TEMP\%USERNAME% which was being reparsed.

Replacing environment variables for current user and for all new users:

reg add "HKEY_CURRENT_USER\Environment" /v TEMP /t "REG_EXPAND_SZ" /d T:\%USERNAME% /f 
reg add "HKEY_CURRENT_USER\Environment" /v TMP /t "REG_EXPAND_SZ" /d T:\%USERNAME% /f 

reg add "HKEY_USERS\.DEFAULT\Environment" /v TEMP /t "REG_EXPAND_SZ" /d T:\^%USERNAME^% /f 
reg add "HKEY_USERS\.DEFAULT\Environment" /v TMP /t "REG_EXPAND_SZ" /d T:\^%USERNAME^% /f 

PS: Please note that the circunflex before the percent (^%USERNAME^%) is necessary for the HKEY_USERS, or else the command would automatically expand the %USERNAME% variable and then all users would have environment variables pointing to T:\Drizin. :-)

After restarting OneNote, everything started working again.

My wife is changing her job, and asked my help to extract all email addresses from her Outlook so that she could send a goodbye email to all her contacts. Outlook has this feature of exporting some fields to CSV but it has some major problems:

  • It does not allow you to export multiple folders, let alone multiple Personal Folders
  • It does not preserve unicode characters
  • It does not remove duplicates

For solving this problem, I wrote a VBA macro to extract all your contacts from all your mail folders. The Macro is easy to use, and you can customize to your needs.

  1. First step is enabling macros in Outlook. On most recent versions the path is something like this:

  2. Fire up the VBA editor by pressing ALT-F11, and expand the existing Project file (on the left) like this:

  3. Paste the following VBA code

    Option Explicit
    Public dict As Variant
    Dim myPath As String, nameEmail As String, strFilename As String
    Dim intLogFile, include As Integer
    Dim pst As Variant, subfolder As Variant, csvFile As Object  
    
    Sub ExtractEmailAddresses()
        Set dict = CreateObject("Scripting.Dictionary")
        'Set CurFolder = Application.ActiveExplorer.CurrentFolder
        Dim myNameSpace As NameSpace
        Set myNameSpace = Application.GetNamespace("MAPI")
    
        Dim fileName As String
        fileName = IIf(Environ$("tmp") <> "", Environ$("tmp"), Environ$("temp")) & "\emails.csv"
    
        If Dir$(fileName) <> "" Then Kill (fileName) '  delete if exists
        Set csvFile = CreateObject("ADODB.Stream")
        csvFile.Type = 2 'Specify stream type - we want To save text/string data.
        csvFile.Charset = "utf-8" 'Specify charset For the source text data.
        csvFile.Open 'Open the stream And write binary data To the object
              
        On Error Resume Next
        For Each pst In myNameSpace.Folders
            include = MsgBox("Include PST " + pst.Name + " (" + pst.Store.FilePath + ") ?", vbYesNo, "Include PST?")
            If (include = vbYes) Then Call Explorefolder(pst, "")
        Next
        csvFile.SaveToFile fileName, 2 'Save binary data To disk
        Shell ("explorer """ & fileName & """")
        Exit Sub
    End Sub
    
    Sub Explorefolder(ByVal folder As folder, ByVal parentPath As String)
        myPath = parentPath & IIf(Len(parentPath) > 0, "\", "") & folder.Name
        Debug.Print "Reading " & myPath
        Call Listemails(folder)
        For Each subfolder In folder.Folders
            Call Explorefolder(subfolder, myPath)
        Next
    End Sub
    
    Sub Listemails(ByVal folder As folder)
        'Dim msg As MailItem
        Dim msg As Object
        Dim rec As Recipient
        Dim Email As String
        Debug.Print "Items: " & folder.Items.Count
        Dim i As Integer
        i = 1
        For Each msg In folder.Items
            If TypeName(msg) = "MailItem" Then
                'Debug.Print "I: " & i
                For Each rec In msg.Recipients
                    'Debug.Print rec.PropertyAccessor.GetProperty("http://schemas.microsoft.com/mapi/proptag/0x39FE001E")
                    'Debug.Print rec.name & ": " & rec.Address
                    nameEmail = rec.Name & " <" & rec.PropertyAccessor.GetProperty("http://schemas.microsoft.com/mapi/proptag/0x39FE001E") & ">," ' comma separated names and email addresses
                    'nameEmail = rec.PropertyAccessor.GetProperty("http://schemas.microsoft.com/mapi/proptag/0x39FE001E") & "," ' comma separated email addresses
                    If (IsNull(nameEmail) Or IsEmpty(nameEmail)) Then nameEmail = rec.Address
                    If (IsNull(nameEmail) Or IsEmpty(nameEmail)) Then nameEmail = rec.Name
                    If Not dict.Exists(nameEmail) Then
                        Call dict.Add(nameEmail, nameEmail)
                        csvFile.WriteText nameEmail & vbCrLf
                    End If
                Next
                i = i + 1
            End If
        Next
    End Sub
    
  4. Position the cursor inside the ExtractEmailAddresses block, and press F5 to run.

    • It will loop through all your PST (and Exchange OST) folders and ask one by one which ones should be processed
    • Each processed PST/OST will recursively loop throught all folders and subfolders, reading recipients for every message
      You can customize as you want. E.g. you can only see emails YOU sent, or only emails explicitely sent to you, etc.
    • By default, it’s only extracting email address, but if you want you can use the commented line that instead uses the concatenation of both name and email address. Please note that you will probably end up with some duplicated address (same email address and different variations of how the name was written).
    • All distinct values are written to a file “emails.csv” inside your %TEMP% folder.
    • After finishing, this CSV file is opened using default associated program. If doesn’t work you can try replacing CSV by TXT.
    • All addresses are already separated by commas, so that you can just copy and paste into your goodbye email.
  5. Celebrate with your work friends, and enjoy your new job!! :-)

Note: Original code was using “Open file For Append”, but it was incorrectly saving non-ANSI characters, so I changed code to write using UTF-8

From ASP.NET Webforms to ASP.NET MVC to GO

For the past 10 years, I’ve used many different blog platforms like (hosted) Blogger, (webforms-based) DotNetNuke, BlogEngine.NET, and (MVC-based) MiniBlog.

More recently, while learning Google’s GO, I’ve decided to migrate my blog to Journey platform, which allows me to use Ghost Themes, is based on Markdown markup language, and uses SQLite as database.

Building and running GO/Journey on Windows was as easy as this:

  1. Download and install GO if you don’t have it.
    Remember to create a base folder for your GO repositories, and set up environment variables GOPATH pointing to it. Remember to add the installation folder (bin subfolder) to the PATH, and set up environment variable GOROOT in case you modified the default installation folder
  2. I had to install a MingW (GNU for Windows) because building SQLite (next step) requires gcc. Install x86_64 if you are running on 64-bits. After installing, add both GO and MinGW to your path:C:\GO\bin , C:\Program Files\mingw-w64\x86_64-5.3.0-posix-seh-rt_v4-rev0\mingw64\bin or C:\MinGW\bin
  3. Download source code and dependencies: go get -u github.com/kabukky/journey
  4. Navigate to Journey folder:cd /d %GOPATH%\src\github.com\kabukky\journey
  5. Build it:go build
  6. Run it:journey.exe

I’ve tried some Ghost templates, and costumized them easily, using different things from each template.

Linux and case-sensitivity

One thing that I don’t like in Linux is that everything is case-sensitive, so most server applications treat URLs as case-sensitive and most databases (like PostgreSQL) treat tables and columns as case-sensitive. The “best-practice” for URLs suggest that you should always use lowercase characters, so that you won’t have problems with someone using the wrong case. The “best-practice” for PostgreSQL suggest that you should never use double-quote on object identifiers, so that they are automatically converted to lowercase, so that you won’t have problems by using the wrong case.

In my opinion this “best-practice” is an ugly workaround for what seems to me like a design problem. I really can’t imagine a case where someone would need two URLs with same address but different cases, or two different tables whose names differ only by some uppercases. I also believe that URLs (and even table names) are much more readable when they contain mixed case, because our brain has been trained for that for decades.
PS: I understand why filesystems are designed being case-sensitive, but I strongly disagree that URLs that should be human-readable (and human-memorable) should follow this. And I’m glad that domain names have been designed correctly in this sense.

Having said that, I made a small modification to Journey so that the search of a post by the slug became case insensitive. If the case doesn’t match exactly, it does a 301 redirect to the canonical url, so that Google will always use the correct mixed-case url. (Someone told me that Ghost also has this feature).

Contact Form

I’ve developed a very simple contact form, creating a page at content\pages\Contact\index.html. Then I modified /server/pages.go to handle not only GETs but also POST:

func InitializePages(router *httptreemux.TreeMux) {
	// For serving standalone projects or pages saved in in content/pages
	router.GET("/pages/*filepath", pagesHandler)
	router.POST("/pages/*filepath", postPagesHandler)
}
func postPagesHandler(w http.ResponseWriter, r *http.Request, params map[string]string) {
	path := filepath.Join(filenames.PagesFilepath, params["filepath"])
	email := r.FormValue("Email")
	message := r.FormValue("Message")
	name := r.FormValue("Name")
	subject := r.FormValue("Subject")
	go sendMail(email, message, name, subject)
	http.ServeFile(w, r, path+"/"+"index.post.html")
	return
}
type SmtpTemplateData struct {
	FromEmail string
	FromName  string
	Subject   string
	Body      string
}
type SMTPSettings struct {
	Username    string
	Password    string
	EmailServer string
	Port        int
}
func sendMail(fromEmail string, message string, fromName string, subject string) {
	const emailTemplate = `From: {{.FromName}}
Subject: {{.Subject}}

{{.Body}}

{{.FromName}} ({{.FromEmail}})
`
	var err error

	//mime := "MIME-version: 1.0;\nContent-Type: text/html; charset=\"UTF-8\";\n";
	mime := "MIME-version: 1.0;\nContent-Type: text/plain; charset=\"UTF-8\";\n"
	//subjectHeader := "Subject: " + encodeRFC2047(subject) + "\n"
	subjectHeader := "Subject: " + subject + "\n"
	to := "To: " + "[email protected]" + "\n"                       // this is just what will be displayed in headers
	message = message + "\n\n" + fromName + " <" + fromEmail + ">" //append real sender to body, since GMAIL automatically fills from
	//msg := []byte(subject + mime + "<html><body></body></html>")
	msg := []byte(subjectHeader + to + mime + "\n" + message) // two line breaks before message body

	emailUser := &SMTPSettings{"[email protected]", "password", "smtp.gmail.com", 587}

	auth := smtp.PlainAuth("",
		emailUser.Username,
		emailUser.Password,
		emailUser.EmailServer)

	err = smtp.SendMail(emailUser.EmailServer+":"+strconv.Itoa(emailUser.Port), // in our case, "smtp.google.com:587"
		auth,
		emailUser.Username,
		[]string{"[email protected]"},
		msg)
	if err != nil {
		log.Print("ERROR: attempting to send a mail ", err)
	}
	return
}

The sendMail() method was based on this sample to send email through Gmail.

Hosting GO on Windows 2012 & IIS 8

I wanted to host the blog on a Windows Server where I already run other websites. Since the server already had other websites running on port 80, I couldn’t simply run journey.exe and use port 80, so I had to use IIS which is already using port 80 and redirecting to different websites according to name-based virtual hosting, which in IIS is called host-header.

The following steps describe how I created a website on IIS for doing the host-header and redirecting requisitions to Journey, hosted on its own port on another IIS process.

First thing is to create a new website. I created a website on IIS using I binded my site to port 80, but using a specific host address (drizin.com.br).

Then, I created a Web.config to redirect all requests that this website catches (at drizin.com.br:80) and redirect them with a reverse proxy to the other port where GO/Journey will listen (8084).

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <system.webServer>
    <rewrite>
      <rules>
        <rule name="Reverse Proxy to GO" stopProcessing="false">
          <match url="^(.*)" />
          <action type="Rewrite" url="http://localhost:8084/{R:1}" />
        </rule>
      </rules>
    </rewrite>
  </system.webServer>
</configuration>

For this reverse-proxy to work you need to install Application Request Routing using Web Platform Installer.

After installing, restart IIS Manager, and enable it here:

Now we need to prepare Journey to run on port 8084. Of course you can just run it in console or as a scheduled task, but you would miss all management features of IIS.

For hosting a standalone server (journey.exe) inside IIS you need to install HTTP Platform Handler using Web Platform Installer.

Then I created a new website for listening on port 8084 to host the GO/Journey process, and created a Web.config to host this external process, and redirect all connections from port 8084 to Journey process using HTTP Platform Handler.

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <system.webServer>
	<!-- need to install HTTP PLATFORM HANDLER 1.2 -->
	<!-- need to give write permissions to IIS_IUSRS, since AppPool runs under ApplicationPoolIdentity -->
        <handlers>
            <add name="httpplatformhandler" path="*" verb="*" modules="httpPlatformHandler" resourceType="Unspecified" />
        </handlers>
        <httpPlatform processPath="E:\RD\Drizin\journey.exe"
        arguments="-log=E:\RD\Drizin\log.txt -http-port=%HTTP_PLATFORM_PORT%"
        startupTimeLimit="60">
        </httpPlatform>
    </system.webServer>
</configuration>

Please note that this second website is bound to port 8084, but the journey.exe is bound to another random port (HTTP_PLATFORM_PORT), and HTTP Platform Handler forwards all 8084 connections to the correct journey port.

I also had to give write permissions to IIS_USRS on the database folder (journey\content\data).

And that’s it. Blog is up and running. :-)