How to build an Android app and deploy an F-Droid repository

Welcome!
These are our notes on how to build an Android app and deploy an F-Droid repository.
This is also a reminder for the future me, when i forget steps (often).
This guide is only possible with the collaboration of the Grey Eminence behind me, whose identity can't be disclosed.

Personal suggestion: if you have the chance and the possibility use a VM to do this.
Android environment can require tweaks that you may not like for your main system, doing this in a VM can help, you can always nuke and redo in case things go nuts.
I may explain my VM setup in a separate guide.

As always, you can do things in different ways, you can manually build things, you can build with F-Droid, etc...
I'll try to be as linear as possible. Let's start!

Setup Android SDK

This is a common prerequisite for both the path (manual build and F-Droid build) so don't skip yet.

Install necessary packages

$ sudo apt install git wget unzip default-jdk-headless

Create a new folder for this project and cd into it

$ mkdir app_builds
$ cd app_builds

Download Android SDK and save it as sdk.zip:

$ wget https://dl.google.com/android/repository/commandlinetools-linux-6858069_latest.zip -O sdk.zip

Note: this is the current link as of writing this guide. This can change in future. Check https://developer.android.com/studio#command-tools for future updates.
Now we have to extract the sdk.zip to android_sdk/cmdline-tools/

Create the directories

$ mkdir -p android_sdk/cmdline-tools/

Extract the files

$ unzip sdk.zip -d android_sdk/cmdline-tools

Command line tools of the Android SDK expects to be in a specific directory, so let's give what it want:

$ mv android_sdk/cmdline-tools/cmdline-tools android_sdk/cmdline-tools/latest

Add ANDROID_HOME environment variable

$ echo "export ANDROID_HOME=~/app_builds/android_sdk" >> ~/.bashrc
$ source ~/.bashrc

Add Java location to PATH

$ echo "export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64" >> ~/.profile
$ echo "export PATH=\$PATH:\$JAVA_HOME/bin" >> ~/.profile
$ source ~/.profile

Now we have to accept the licenses

$ ./android_sdk/cmdline-tools/latest/bin/sdkmanager --licenses

Once accepted see the package you can install with the command:

$ ./android_sdk/cmdline-tools/latest/bin/sdkmanager --list

At the moment of writing, the latest build tool version is 30.0.3. We need it to build the apps, so let's download it with:

$ ./android_sdk/cmdline-tools/latest/bin/sdkmanager --install "build-tools;30.0.3"

Prerequisites are done. Now, if you want to manual build apps, keep reading. If you want to build apps with F-Droid skip to F-Droid Server Chapter.

Manual Android app build

If you never built an app before, i suggest you to start with manually building an app. This will let you learn a lot of things while troubleshooting errors you will encounter.

Let's take Fedilab as example for build. The procedure is very similar with almost any app.
Clone Fedilab’s repository

$ git clone https://framagit.org/tom79/fedilab.git fedilab

Go to the cloned folder and build!

$ cd fedilab
$ ./gradlew assembleFdroid

I wrote assembleFdroid after the command. It's not the same for every app, for example for Blabber.im is assembleGit.
Check the build.gradle file in your app's git repo to see how is called the flavour.
Different flavour leads to different results: some flavours for example requires Google dependencies we may not like...
It may happens the app has no flavours. In this case it's sufficient to execute

$ ./gradlew assemble

Signature Key

This is needed just the first time.
We need to generate our key to sign APK.

$ cd ~/app_builds/
$ keytool -genkey -v -keystore NAME.keystore -keyalg RSA -keysize 4096 -validity 10000 -alias NAME

It will ask you some infos, some can be left blank. After we have to convert it to PKCS12 format to fix the warning that just appeared.

$ keytool -importkeystore -srckeystore NAME.keystore -destkeystore NAME.keystore -deststoretype pkcs12

To make it quicker to sign the APKs i suggest you to add an alias in the end of .bashrc file.

$ nano ~/.bashrc

Paste and edit this

alias apksigner="~/app_builds/android_sdk/build-tools/30.0.3/apksigner sign --ks ~/app_builds/NAME.keystore --ks-key-alias NAME"

Then

$ source ~/.bashrc

Sign APKs

Now sign the apk (continuing the Fedilab example)

cp app/build/outputs/apk/fdroid/release/app-fdroid-release-unsigned.apk fedilab.apk
~/app_builds/android_sdk/build-tools/30.0.3/apksigner sign --ks /YOURALIAS.keystore --ks-key-alias YOURALIAS fedilab.apk

Or if you followed my suggestion of the alias just

$ apksigner fedilab.apk

Done! Copy it to your phone and install! Of course, since your signature is different from the F-Droid one (if you installed already it from their repo) you need to uninstall that first.

Daily

Go in every folder for each app you want to build, get the latest code, build and sign.

$ cd fedilab
$ git pull
$ ./gradlew assembleFdroid
$ cp app/build/outputs/apk/fdroid/release/app-fdroid-release-unsigned.apk fedilab.apk
$ apksigner fedilab.apk

As you can see this can take some time if you have a lot of apps...

F-Droid Server

Instead of manual build each app, you could automatise things a bit and use F-Droid server.
This requires more initial setup, but hopefully you gain time daily, especially if you want to build a lot of apps.

Setup

We need few more packages

$ sudo apt install python3-pip curl rsync

If you aren't already there, move yourself into app_builds folder

$ cd app_builds

Clone the fdroidserver repo

$ git clone https://gitlab.com/fdroid/fdroidserver.git _fdroidserver

Install fdroidserver using pip

$ pip3 install -e _fdroidserver

Now run the command fdroid to check if it works. If not, try this

$ source ~/.profile

Now create a folder for this and cd into it

$ mkdir fdroid
$ cd fdroid

Initialize a fdroid working directory

$ fdroid init

Previous command will create a keystore, which is used for signing your F-Droid repository. We are going to remove it and manually create a new one. If you generated the key in the manual steps you can use that and skip this step.

$ rm keystore.p12 keytool -genkey -keystore keystore.p12 -alias NAME -keyalg RSA -keysize 4096 -sigalg SHA256withRSA -validity 10000 -storetype pkcs12 -dname "CN=NAME, OU=F-Droid" -J-Duser.language=en

Replace NAME with a name you like.
It will ask you for a password. Type a good password.
Edit the config.yml and update below values in it nano config.yml

keystorepass: #password you used in previous step
keypass: #password you used in previous step
repo_keyalias: NAME #Replace NAME with the same value you used when creating the keystore
keydname: CN=NAME, OU=F-Droid #Replace NAME with the same value you used when creating the keystore

Save and exit the text editor (press CTRL+X then press Y ). We're done!

Build

First we need to tell the F-Droid what to build. You have 2 ways.

The easiest is fetch a metadata configuration from F-Droid Data repo of your app (if exists there).
Copy metadata in fdroid/metadata folder. The metadata file name is something like de.pixart.messenger.yml
Then run this in fdroid folder

$ fdroid checkupdates --allow-dirty --verbose --auto

It will clone the repo to fdroid/build folder.

OR

Use this command to import an app:

$ fdroid import --url=http://address.of.project

It will create a basic metadata file and you can edit it yourself.

Now let's trigger the build with:

$ fdroid build --latest --no-tarball --verbose de.pixart.messenger

If you have multiple apps you can build them all with --all instead of app codename. If the app is already build (no version change), it skips the build of course.

$ fdroid build --latest --no-tarball --verbose --all

If it says "Build successful", congrats! You made it!
If not, check what it says. Usually you just have to install some SDK packages. Check Troubleshoot Chapter to fix some error i encountered so far in my experience.

Sign

Similar to the manual build, we have to sign APKs before we made them available.
The apk you built are in the unsigned/ folder.

$ ../android_sdk/build-tools/30.0.3/apksigner sign --ks keystore.p12 --ks-key-alias NAME unsigned/NAMEOFTHE.apk

Replace NAME with a name you used to create the key and NAMEOFTHE.apk with the name of the file. Insert the password and done!. Of course if you added an alias for apksigner it's way easier. See above

Prepare for deploy

This is needed just the first time.
Place an icon of your choice into /fdroid folder.
I created a cool QR code with a logo inside with the url of my repo and my avatar here: https://www.logodesign.net/qrcode-generator

Add or update these in config.yml

make_current_version_link: false
deploy_process_logs: false
repo_url: https://codeberg.org/silkevicious/apkrepo/raw/branch/master/fdroid/repo/
repo_name: My F-Droid Repo
repo_description: This is a repository of apps to be used with F-Droid.
archive_older: 0

I decided to make available via git my repo. You can also make available via normal website (i may cover this in future).
Edit config.yml Add or update these in config.yml

git_mirror_size_limit: 100MB
identity_file: ~/.ssh/ssh_key_for_git
servergitmirrors: git@codeberg.org:silkevicious/apkrepo.git

Deploy

Place APKs fdroid/repo, from your manually build folder or from the unsigned folder, then run in fdroid folder

$ cd fdroid
$ fdroid update

You should run with -c if you haven't copied metadata from fdroid so it can add a skeleton of metadatas to edit.

$ fdroid update -c

And now

$ fdroid deploy

If you add your repo to fdroid you can now download and install your apps. Great job!

Daily

Go in fdroid folder, build, sign and deploy

$ sudo apt update && apt upgrade -y
$ cd ~/app_builds/fdroid
$ fdroid checkupdates --allow-dirty --auto && fdroid build --latest --no-tarball --verbose --all

then sign if there are new builds (this is the only boring move), move to repo folder and

$ fdroid update && fdroid deploy

Sometimes check also if the fdroidserver got updates

$ cd ~/app_builds/_fdroidserver
$ git pull

Thanks to the -e parameter while installing it we don't have to do anything else.

Troubleshooting

Packages missing

Some apps require extra packages to be installed on the system. For example npm is required by Tutanota, mercurial by Klar and maven by Seafile. There can be others, depending what you build.

$ sudo apt install npm mercurial maven

NDK missing

Some apps require specific versions of NDK.

$ ./android_sdk/cmdline-tools/latest/bin/sdkmanager --list
$ ./android_sdk/cmdline-tools/latest/bin/sdkmanager --install "ndk;21.3.6528147"

Also add or update these in config.yml

ndk_paths:
r21d: $ANDROID_HOME/ndk/21.3.6528147/

For example i have some ndk in my config:

ndk_paths:
  r17c: $ANDROID_HOME/ndk/17.2.4988734/
  r20b: $ANDROID_HOME/ndk/20.1.5948944/
  r21: $ANDROID_HOME/ndk/21.0.6113669/
  r21d: $ANDROID_HOME/ndk/21.3.6528147/
  r22: $ANDROID_HOME/ndk/22.0.7026061/

Java errors

Some apps gave me some java errors. They required the old Java 8. This instruction are for Debian, but with few edits are ok for any other distro.

$ sudo sh -c "echo 'deb http://deb.debian.org/debian/ stretch main' >> /etc/apt/sources.list.d/stretch.list"
$ sudo sh -c "echo '\ndeb http://security.debian.org/debian-security/ stretch/updates main' >> /etc/apt/sources.list.d/stretch.list"
$ sudo sh -c "printf 'Package: *\nPin: release a=stretch\nPin-Priority: 90\n' >> /etc/apt/preferences.d/limit-stretch"
$ sudo apt update && sudo apt install openjdk-8-jdk-headless

and edit ~/.profile

JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64

and then

$ source ~/.profile

Also add or update these in config.yml

java_paths:
  8: /usr/lib/jvm/java-8-openjdk-amd64/
  11: /usr/lib/jvm/java-11-openjdk-amd64/

Warning or error about permission.

Happened to me after i nuked and rebuilt a VM. I moved files back and forth. Check permission of ssh and config.yml keys.

$ chmod 0600 config.yml

Can't deploy on git because permission error

Happened to me the first time, with maiden VM. start ssh agent and add your ssh key

$ eval "$(ssh-agent -s)"
$ ssh-add -K /Users/you/.ssh/id_rsa

You have to put your git host in the known hosts otherwise it will fail

$ git clone git@codeberg.org:silkevicious/apkrepo.git test

You may also want to config globally your email and name (not mandatory):

$ git config --global user.email "your@email.com"
$ git config --global user.name "Your Name"