In a previous article I installed a #Mastodon #Glitch Edition intended to become my techie instance, moving there my bots from the previous #Firefish instance. The final step was to migrate my personal nerd account and this article was supposed to be a walk through… just that it’s not. It is a log about my adventure and failures to move an admin account away from Firefish, to execute a successful #fediverse account #migration against all odds.

So bear with me, grab a sit and a drink, and let me explain you how (not) to do a migration between fediverse instances.

Overview

This article is basically a walkthrough the steps to migrate an account from Firefish to Mastodon. It’s just that I found nowhere references to issues relating the fact that the account is an admin, so in a given section things get complicated with several failed approaches and a final solution.

I’ve split the guide in the following sections:

0. Assumptions and requirements
1. Export from Firefish
2. Import to Mastodon
3. Set up the new account as an alias for the old account
4. Move from the old account
   • 4.1. What it should be
   • 4.2. Failed workaround 1: Downgrade the account
   • 4.3. Failed workaround 2: Manually set the MovedToUri user’s parameter
   • 4.4. Succeeded workaround 3: Update the Firefish code
5. Verify that the new account has the old account’s followers
6. Wrapping up

0. Assumptions and requirements

This article is still technical even the steps are designed to be executed just from the instance’s application. Therefore, allow me to still mention some points regarding the set up

• The old instance is a Firefish under Docker in a Raspberry Pi 4.
• The new instance is a Mastodon Glitch Edition under Docker in a Raspberry Pi 4, installed as explained previously in Install Mastodon Glitch Edition on a Raspberry Pi 4
• A bit aside for the purpose of this article, but both Raspberry Pi machines are part of the same local network, receiving the traffic through a Reverse Proxy in a separated Raspberry Pi machine as explained in Set up a Reverse Proxy in a Raspberry Pi with Apache which also works as a DNS server as explained in Quick DNS server on a Raspberry Pi.

A little more into the purpose of this article, it is important to mention that:

• The origin account (the one to be migrated from) is called xavi and it’s the Firefish admin account: @xavi@social.devnamic.com
• The target account (the one to be migrated to) is called xavi and it’s the Mastodon admin account: @xavi@social.arnaus.net
• Just to repeat myself, the target account already exists and is up and running!

1. Export from Firefish

The very first step is to pull all the possible data that we can get from our current Firefish instance.

We move to Settings > Import/Export Data and for each section, select Export and click the Export button:

• All Posts (won’t be imported, just to have them)
• Followed users
• User lists
• Muted users
• Blocked users

Firefish won’t give the files to you directly. They may be heavy to put together by the instance and instead it publish it for you in your Drive space once the work is completed.

So, after requesting all the data above and short waiting period, you should be able to download them from the section Drive, from your side bar.

2. Import to Mastodon

On the shiny new Mastodon account, we can import some of the data that we exported from Firefish.

We move to Preferences > Import and Export > Import and for each Import type from the drop down menu, select a CSV Data file, select if you want to Merge it or Overwrite and press the button Upload. If you don’t know what to choose, take Merge as it’s not destructive.

• Following list (this is Followed users in Firefish)
• Bookmarks (not exportable from Firefish)
• Lists (this is User lists in Firefish)
• Muting list (this is Muted users in Firefish)
• Blocking list (this is Blocked users in Firefish)
• Domain blocking list (not exportable from Firefish)

The imported files appear listed below the uploading form. They are applied straight away, so don’t get scared if right after importing the Following list some people follow back without having actually migrated the account.

3. Set up the new account as an alias for the old account

This is the first step for the actual account migration. We need to tell the target account that it’s going to receive the forwarding of another one.

In the Mastodon instance, we go to Preferences > Account > Account settings > Moving from a different account

1. Click on the link “create an account alias”. A new form appears asking for the “Handle of the old account"
2. Enter the old account handle in the format xavi@social.devnamic.com
3. Click on Create Alias button A message appears saying that “Successfully created a new alias. You can now initiate the move from the old account."

4. Move from the old account

In this step we’re meant to achieve 2 main points:

• We want to set the old account to forward all new interactions to the new account. We want that all visitors get to know that the account has moved, and therefore the old account rests inactive and dormant. It won’t let new posts to be published and will preset a message in the profile page.
• We want that the followers get updated by this move and that they update their records. We want that when we go to their following, the new account has replaced the old account in their list.

4.1. What it should be

So we go to the Firefish instance and there we move to Settings > Migration > Move current account to new account.

1. Enter the new account handle in the box labeled as “Account you’re moving to” in the format xavi@social.arnaus.net
2. Click on Move Account! button
3. Confirm the modal that appears

But, as this account is an admin, a nasty error modal appears:

🔴 Admins cant migrate. 4362e8dc-731f-4ad8-a694-be2a88922a24

Let’s see how can we fix it.

4.2. Failed workaround 1: Downgrade the account

The first thought I had was to create a secondary Admin account and then downgrade my admin account xavi so that the migration can happen.

At the very end of the main README.md of the Firefish source code we can find a Tips & Tricks that mention how to create another admin account:

• To add another admin account:
  • Go to the user's page > 3 Dots > About > Moderation > turn on "Moderator"
  • Go back to Overview > click the clipboard icon next to the ID
  • Run psql -d firefish (or whatever the database name is)
  • Run UPDATE "user" SET "isAdmin" = true WHERE id='999999'; (replace 999999 with the copied ID)
  • Restart your Firefish server

Well, this actually did not work as is. I don’t know if it’s due to having the instance in Docker or because this is a copy & paste from Calckey’s original README.md file and things changed meanwhile, but here are the variations:

4.2.1. Enter into the Postgres DB

Understanding that we’re already SSH-ed into the machine and moved into the directory that holds the instance:

1. Get the data that our instance uses to connect to the DB.

   cat .config/docker.env

   It will output something like:

   POSTGRES_PASSWORD=firefish-pass
   POSTGRES_USER=firefish-user
   POSTGRES_DB=firefish

2. Log into the docker container. In my case the [container_id] is firefish_db:

   docker exec -it [container_id] bash

3. Log into the DB specifying the username, which is the value of POSTGRES_USER in the point 1:

   psql -d firefish -U firefish-user

4.2.2. Query to update the user table setting up the admin

Here I lack of Postgres knowledge. I come only with SQL knowledge that I collected with MySQL and others, but not specifically with Postgres.

Once the new admin user is created, it is still only a normal user. Then, by doing the mentioned step it is moved to moderator, which is the maximum it can get from within the Firefish control panel

Go to the user's page > 3 Dots > About > Moderation > turn on "Moderator"

With the next step we should have copied the hash that works as ID. Let’s say it is 9p322wervu83w7wq

Go back to Overview > click the clipboard icon next to the ID

And then, once we followed the step 4.2.2. above we’re ready to run the following query. Note that I modified it a bit, as the one in the README.md did not work for me:

update public.user set "isAdmin" = true where id = '9p322wervu83w7wq';

Also, this is done to have an admin account that replaces my xavi account that I intend to downgrade, so from within the control panel, I go to Users and select my xavi user, and then I get the hash Id from it with the same procedure:

Go back to Overview > click the clipboard icon next to the ID

Let’s say it’s 9kbgrf13tumjcz7b. Then in the Postgres terminal connection, I run the following command to downgrade it:

update public.user set "isAdmin" = false where id = '9kbgrf13tumjcz7b';

4.2.3. Restart the Firefish Server

Yeah, nice. Do it!. I have no idea how to restart a NodeJS app that runs inside a Docker container. I could not find any command to restart the server, or stop it. Only to start it. I could see in the package.json file from the original code that there are scripts to perform some tasks, but...

• I did not really wanted to rebuild the NodeJS app, just restart the server
• The scripts for cleaning the app left the instance unusable

So at the end of the day, the only thing I could think of was:

Stop the docker container set

docker composer down

Start the docker container set

docker composer up -d

… so I did.

4.2.4. Result for Workaround 1: Fail ❌

Once the app restarted I could log in back into the instance.

• The new admin user is presented as moderator everywhere. Nowhere says it is an admin account. Still, it can enter into the control panel and perform all tasks, I could not spot a difference to an admin account.
• My user lost the admin privileges, it does not say anywhere that it is an admin, and in fact it lost access to the control panel.

Great, so I go to the Settings > Migration > Move current account to new account, set the new account, click the button, but the same message appears:

🔴 Admins cant migrate. 4362e8dc-731f-4ad8-a694-be2a88922a24

I could not figure out where does it say still that my user is an admin. I suspect that there is any app cache, or that restarting the container does not really restart the NodeJS app. Anyways, nothing really moved anymore in this path.

4.3. Failed workaround 2: Manually set the MovedToUri user’s parameter

Well, so in my enormous un-knowledge about Firefish, ActivityPub and how these account migrations work, my next thought was Ok then I have access to the DB, so I am going to set the account as moved manually and the instances will then redirect

I discovered that in Control Panel > Users > [username] > Raw tab one can see the parameters of the user. This has to be a representation of one or more tables in the DB. There is a field called movedToUri. Let’s pull the cord.

So there I go, following the point 4.2.1. above to connect to the DB and checking for the users table:

select * from public.user where id = '9kbgrf13tumjcz7b';

Aha, turns out that there is actually a movedToUri field. I check for another user that I already migrated successfuly and turns out that it expects a URL to the new user in the new instance. Then I emulate the action it and update it directly:

update public.user set "movedToUri" = 'https://social.arnaus.net/users/xavi' where id = '9kbgrf13tumjcz7b';

Then I “restarted” as explained in 4.2.3. and I found out that my account displays in the Profile page a note in the top telling that

User has moved to a new account: @xavi@social.arnaus.net

Great! Really? Well actually not that great.

One of the benefits of the migration feature is that the process should move your followers from the old account to the new one. It is actually happening by looping through the old account’s followers and send an update to every one of them, as I found a quote here:

Once a forwarding address is configured, the profile is locked (won’t accept any new content) and you have the option of automatically migrating followers. This is done by the old instance sending each follower a Move notification, forcing him or her to unsubscribe from the current profile and follow the new profile instead.

Great, so with the direct DB update I just lost the opportunity to make the code to perform the Move notification.

4.3.1. Send the Move notification manually

Ok, so if a Move notification has to be sent to all of the followers, then let’s do it. It can’t be so difficult, it’s all about JSON bodies and POST HTTP requests. I’ll make a Python script for that.

I found this nice explanation here and the official ‘Move’ documentation in ActivityBub and I started crafting a Python script that loops through the followers, builds the JSON and sends the POST request.

I quickly ran into the security issue. We’re all SSL servers, and not only that, the messages go signed with the Key Pairs from the sending actor. The target server kept answering me with HTTP 401 unauthorised. I learned in record time about PEM files, Public and Private keys, signature headers… I updated my script but got blocked being unable to export the PEM pair keys from the Firefish DB into a pair of PEM files so that I can read them with my script.

4.3.2. Result for Workaround 2: Fail ❌

At the time I already had invested a bunch of days in a failed migration and needed to pause my brain from all of this. I posted a help request into my new nerd Mastodon instance (of course, with 4 followers by then, expecting the best), but I left the project a bit abandoned.

Next steps would have been connecting the Python script directly to the Postgres DB and get the data from there. When I used a controlled PEM pair of files it worked but of course, the instance said that this message is not accepted as the signature does not match with the supposed actor in the supposed origin. So yeah, my script worked good but the security layer was failing.

4.4. Succeeded workaround 3: Update the Firefish code

Then the magic of internet happened. A Mastodon user called KalenXI answered in the thread where I’m following up with this topic:

@xavi I ran into this issue myself when trying to migrate from my single-user instance of Firefish. I fixed it by just disabling the isAdmin check in the API source code (https://git.joinfirefish.org/firefish/firefish/-/blob/develop/packages/backend/src/server/api/endpoints/i/move.ts#L95) And then I was able to initiate a move.

How stupid am I? So much effort and I did not think about simply removing the check in the code? My gosh.

4.4.1. Current status of the instance: too many tests.

So the instance right now has a bunch of migrated users. The user that I wanted to migrate conserving the followers is half-behaving as an admin, half-behaving as already migrated (does not let me post new content and sows as migrated in the Profile).

I tried to revert back to the admin unmigrated state but for any reason it did not showed any effect. So everything is half cooked. Let’s see where do we get to.

4.4.2. Updating the code and applying

I took a look at the code, where KalenXI pointed me to:

if (!ps.moveToAccount) throw new ApiError(meta.errors.noSuchMoveTarget);
if (user.isAdmin) throw new ApiError(meta.errors.adminForbidden);
if (user.movedToUri) throw new ApiError(meta.errors.alreadyMoved);

These are the 3 checks that the moving function has at the very beginning. Mmm... so I could just comment out the isAdmin and the movedToUri, so even I have both half done it would not matter...

So wait, one more time, the instance runs under Docker, so I can’t simply change the code. I can enter into the container and change the code, but I guess that I will still need to restart the server.

You know what? I have the time, so let’s do it the long way: I’ll change the code and generate a new Docker image.

1. Update the code simply in the directory of the host:

   nano packages/backend/src/server/api/endpoints/i/move.ts

2. Generate a new image with the code that is currently in the host (should be exactly the same as the last image I generated. I do not git pull so I don’t have to face migrations nor code changes)

   docker build . -t firefish:20240205 -f Dockerfile

   It takes a good hour…

3. Update the docker-compose.yml file so that the web container loads the image we just built firefish:20240205

   nano docker-compose.yml

4. Restart the container

   docker composer up -d

5. Check that everything is correct in the logs

   docker logs -f firefish_web

Ok… the instance seems to be up and running...

4.3.2. Result for Workaround 3: Succeed ✅

So we come back to the Instance into my user and move to Settings > Migration > Move current account to new account

1. Enter the new account handle in the box labeled as “Account you’re moving to” in the format xavi@social.arnaus.net
2. Click on Move Account! button
3. Confirm the modal that appears

… and a big Checkbox appears in the screen. Not really… And I see the logs starting to scroll like crazy. Great! It is actually working! Here one of the followers:

query is slow: SELECT host FROM "instance" "instance" WHERE "instance"."host" in ($1) AND ("instance"."isSuspended") -- PARAMETERS: ["mstdn.chrisalemany.ca"]
execution time: 335
INFO 2  [remote ap] Updating the Person: https://mstdn.chrisalemany.ca/users/chris
INFO 2  [remote ap] Creating the Image: https://mstdn.chrisalemany.ca/system/acc...50ae.jpeg
INFO 2  [remote ap] Creating the Image: https://mstdn.chrisalemany.ca/system/acc...3b88.jpeg
INFO 2  [drive register]    file with same hash is found: 9kqc...mcuk
DONE 2  [drive downloader]  Got: 9kqc...mcuk
INFO 2  [download]  Downloading https://mstdn.chrisalemany.ca/system/acc...50ae.jpeg ...
INFO 2  [download]  Downloading https://mstdn.chrisalemany.ca/system/acc...3b88.jpeg ...
INFO 2  [drive register]    {"size":50834,"md5":"89fabd3de...913522","type":{"mime":"image/jpeg","ext":"jpg"},"width":400,"height":400,"blurhash":"yFE...niVrM_9Er=sEI:s8s9X...oxazkB","sensitive":false,"porn":false,"warnings":[]}

5. Verify that the new account has the old account’s followers

I went into the new account and saw:

• I had around 30 notifications of new followers
• I the followers counter on my profile got updated!

It worked! 🎉

6. Wrapping up

The learning I get from this experience is that the process is well thought for normal users. They are meant to follow some steps and the account should be moved successfully between instances. Unless you’re an admin. At least, in Firefish you can’t move out if you’re an admin. Maybe in my next installation I am going to advocate to have an admin proper account and leave my xavi account as a normal one.

In the other hand, I am very happy that this story ended good, all thanks to a guy that just put together an answer from his own experience and gave me just the line I needed to try another approach. Which on top of it I find myself stupid not to have tried this approach in the first place.

I remember going through the code to try to find how the system creates the notification signature and copy the steps, but honestly I think I assumed that the process would have been way more protected, not a single line that could be commented out and it just works.

Anyways, good is what good ends, so let’s celebrate and focus in the next project.

Previous Post Next Post