UPDATES

Migration of a Dynamic Web site to a Static Web site


Photograph by Sai Abhinivesh Burla on Unsplash

Within the earlier two articles (Migration from Traditional Internet hosting to Serverless and Migration of a Multiplayer Sport from Hosted to Serverless) I’ve launched you to my plan of migrating away from my devoted server to a totally serverless infrastructure. This time I wish to go into the plan in additional element.



Why Migrate?!

Basically, I count on from this migration:

  • A extra clear code base (lastly I can clear up my stuff, perhaps take away one thing and modernize another elements)
  • No extra FTP or messy / unclear deployments – the whole lot needs to be dealt with by CI/CD pipelines
  • Price discount; sounds bizarre, however the very last thing I wish to have is a value improve (right this moment it is about 30 € monthly for the internet hosting and my objective is to deliver this under or near 10 € – word: I pay rather more for domains and these prices are usually not included right here as they are going to stay the identical).

There’s a little bit of background to this: Having my very own devoted server is one thing I used to be initially was completely happy about, nonetheless, over all of the years the burden of correctly sustaining this machine was a bit too excessive. I’ve fairly some issues on my plate and coping with the (software-side) of a devoted server was at all times on the underside a part of my ToDo checklist.

Over all of the years the thought to maneuver all elements of my server to the cloud definitely began to attraction. Nonetheless, one factor that at all times stood in the best way was e mail. I would like (and want) my e mail to be additionally hosted, and proper now the one choice can be to arrange a digital machine (VM) at some supplier… which is what I wish to keep away from badly! Not solely price VMs on the in style cloud suppliers greater than my present devoted internet hosting, that additionally would require me to be a mail admin. And belief me – the very last thing I wish to be is a mail admin. It is exhausting.

Now I wanted to search for an answer as – for the second time since I’ve a devoted server – the configuration I take advantage of is phased out at my internet hosting supplier. Like the primary time there is no such thing as a comparable configuration out there, so I must get one thing extra beefy (and costlier). This time, nonetheless, I am going to skip this necessary improve; I transfer to the cloud.

That is already ultimate – as proven under (German “Kündigung” means “cancellation” or “termination” of the service):

Quitting the dedicated server

Nonetheless, I first want to resolve the issue of e mail.



Fixing E mail

I did a little bit of analysis on the out there mail suppliers:

  • Google, Microsoft and so on.
  • Proton Mail
  • Zoho Mail
  • Posteo
  • Yandex
  • Fastmail

In the long run I went with Fastmail. Why? Fairly easy – it just about solves the core drawback of my Cloud migration in a method that makes the entire migration work with out a lot bother. How?!

Fastmail provides the opportunity of utilizing customized domains. This fashion you do not get only a consumer@fastmail.com deal with, however reasonably a chief@yourdomain.com. Whereas most different mail suppliers additionally provide customized area help Fastmail has for a price range of $5 a customized DNS on high of it – supporting as much as 100 customized domains (sufficient for me) with free configuration. That is proper – it is primarily a $5 DNS service with e mail.

However Fastmail doesn’t cease there. One different drawback can be that I register a customized area similar to florian-rappl.de however then I may solely set a CNAME for www.florian-rappl.de leaving florian-rappl.de undefined. Whereas most browsers will deal with this state of affairs very properly (with an auto-completion of the www subdomain) there’s one browser that makes bother right here; however you’d must get on a Safari to seek out it. How does Fastmail clear up this root area drawback? With static web sites!

On Fastmail you may create a redirect for the customized area. So I can simply make https://florian-rappl.de reply mechanically with a 301 (completely moved) to https://www.florian-rappl.de. Ah and, in fact, the web site is mechanically HTTPs protected by a Let’s Encrypt certificates.

With e mail solved it is time to look the place I have been and the place I wish to be from an structure perspective.



Earlier Structure

Beforehand, the whole lot was hosted on a devoted server. The entire DNS service, mail and web sites have all been served from the identical machine, which additionally hosted the database. To deliver up to date content material to the server FTP has been used. Any type of configuration was carried out manually.

The next diagram illustrates this.

Previous architecture

There are three facets that I wish to see shifting to a brand new structure:

  • It needs to be extra versatile, i.e., leaving room to make use of different languages and paradigms
  • It needs to be extra environment friendly, i.e., having the ability to serve my web page (or another challenge) even quicker
  • It needs to be simpler for me to know and preserve

All this needs to be fulfilled with out going over the earlier price range. Ideally, it needs to be even cheaper than beforehand. Simply as a quantity: My web page has round 40k views monthly – so that is the load that I count on and that the brand new structure ought to have the ability to deal with this type of load properly.

Under I’ve sketched how I envision the brand new structure. Configuration and updates on this scheme are all carried out by way of CI/CD. The mail supplier additionally has an in depth API permitting me to ship and obtain emails mechanically / have an AI assistant in between (extra on that side later).

Anticipated architecture

That is, in fact, fairly high-level. The precise particulars, i.e., the place every web site that I’ve is rolled out and the way these containers join in all element are to be decided in a extra low-level diagram.

Basically I can’t take away any web site. I’m an enormous proponent of dependable URLs, i.e., I can’t drop or change URLs deliberately. Something that was on the internet final yr ought to nonetheless be on the internet. One factor, nonetheless, is that not the whole lot wants to stay as-is, however I’m free to vary (or simplify) if it retains the earlier content material (principally) intact.

One instance is a web site I did for a lecture I gave on software program design patterns. For this I made a decision to make a dynamic to static conversion.



Dynamic to Static Conversion

For patterns.florian-rappl.de I needed to take away the dynamic half. This was a bit troublesome, as the entire web page (each presentation and slide) has been generated dynamically by a customized CMS.

As a substitute, what I ended up doing is using AngleSharp for reworking the present (dynamic) web sites into static information. I’ve saved them on disk and made them able to be served statically.

The next script was used to get the preliminary obtain of the static pages:

Listing<string> downloadedUrls = new ();

async Process Important()
{
    var url = "https://patterns.florian-rappl.de/";
    var goal = "~/code/florian-rappl-patterns/public";
    await DownloadPage(url, goal);
}

void CreateIfNotExists(string dir)
{
    if (!Listing.Exists(dir))
    {
        Listing.CreateDirectory(dir);
    }
}

async Process DownloadAsset(Url url, string targetDir)
{
    if (!downloadedUrls.Incorporates(url.Href))
    {
        var file = Path.Mix(targetDir, url.Path);
        var dir = Path.GetDirectoryName(file);
        CreateIfNotExists(dir);

        var shopper = new HttpClient();
        utilizing var stream = await shopper.GetStreamAsync(url.Href);
        utilizing var fs = File.Create(file);
        await stream.CopyToAsync(fs);
    }
}

async Process DownloadPage(Url url, string targetDir)
{
    // Do not obtain these - they would not be helpful and shall be eliminated
    if (url.Path.StartsWith("Account") || url.Path.StartsWith("Slides"))
    {
        return;
    }

    // solely obtain web sites inside the origin
    if (!downloadedUrls.Incorporates(url.Href))
    {
        var dir = Path.Mix(targetDir, url.Path);
        var indexPath = Path.Mix(dir, "index.html");
        var config = Configuration.Default.WithRequesters().WithDefaultLoader();
        var context = BrowsingContext.New(config);
        var doc = await context.OpenAsync(url);

        CreateIfNotExists(dir);
        File.WriteAllText(indexPath, doc.Supply.Textual content);

        downloadedUrls.Add(url.Href);

        // obtain all stylesheets
        foreach (var hyperlink in doc.QuerySelectorAll<IHtmlLinkElement>("hyperlink[href]"))
        {
            var href = hyperlink.Href;
            await DownloadAsset(href, targetDir);
        }

        // obtain all scripts
        foreach (var hyperlink in doc.QuerySelectorAll<IHtmlScriptElement>("script[src]"))
        {
            var href = hyperlink.Supply;
            await DownloadAsset(new Url(href, url.Href), targetDir);
        }

        // obtain all photographs
        foreach (var hyperlink in doc.QuerySelectorAll<IHtmlImageElement>("img[src]"))
        {
            var href = hyperlink.Supply;
            await DownloadAsset(href, targetDir);
        }

        // comply with all hyperlinks
        foreach (var anchor in doc.QuerySelectorAll<IHtmlAnchorElement>("a"))
        {
            var href = anchor.Href;

            if (href.StartsWith(url.Origin))
            {
                await DownloadPage(href, targetDir);
            }
        }
    }
}
Enter fullscreen mode

Exit fullscreen mode

After the script was utilized we’ve got all of the out there pages downloaded and out there for being served in a static web site.

Utilizing a search and substitute I’ve additionally added just a few enhancements similar to utilizing a file like jquery.js as a substitute of the beforehand given URL /bundles/jquery?someid, which was initially resulting in a bundle endpoint that carried out some MVC magic.

One other factor I did by way of a search and substitute was to remodel the URLs for the UML diagrams (normally one thing like /diagrams/1d830940-8feb-4c70-b355-b5370cfcd825) to a correct SVG reference (/diagrams/1d830940-8feb-4c70-b355-b5370cfcd825.svg). With a bit extra time make investments I may have additionally renamed that correctly, e.g., /diagrams/mvc-pattern.svg, however having the correct extension is sweet sufficient for now.

The results of the static-ification of the web site is seen under. Full URLs remodeled right into a folder construction with an index.html. Absolutely, a little bit of a nicer transformation would use one thing like Astro with correct re-use, nonetheless, the trouble for such a metamorphosis would have been fairly greater. If the web site would nonetheless be actively used or I might envision some development right here within the following years I probably would make investments the time, however proper now it doesn’t appear to be wanted.

Structure after static-ification

The deployment is completed by way of an Azure Pipeline:

set off:
- grasp

pool:
  vmImage: ubuntu-latest

variables:
- group: deployment-tokens

steps:
- activity: AzureStaticWebApp@0
  inputs:
    app_location: '/public'
    api_location: '/api'
    skip_app_build: true
    azure_static_web_apps_api_token: '$(patterns-token)'
Enter fullscreen mode

Exit fullscreen mode

The deployment is completed to Azure Static Internet App. That is excellent for the state of affairs at hand; a principally static web site with just a few APIs (on this case completely used for the search). Observe that the free tier of Azure SWA is used, i.e., that is one other space of my web site that’s now working at primarily zero price.



Conclusion

It runs – quicker and extra price environment friendly (for the given subdomain no further prices will happen). The essential half was to establish a method of offering the content material in a mode that matches its objective finest.

Making the patterns web site static was the best selection – the dynamic nature of this web page was not longer required. All the fabric has been created and the person HTML information are ample for maintaining the earlier consumer expertise alive.

Within the subsequent put up I am going to look into the migration of a recreation (Mario 5) with its degree editor and backend API.

Presently, the devoted server continues to be operational – however I want to complete the migration till the tip of the yr.

Leave a Reply

Your email address will not be published. Required fields are marked *