WAToAndroid (1.0) -> WhatsApp Migration Tools (2.0)

Author: ferferga

README.md

@@ -1,13 +1,87 @@
-# WAToAndroid
+# WhatsApp Migration Tools
 
-This python script is my own attempt to move my WhatsApp messages, as I needed a quick but custom way to move my messages and calls to a new phone.
+WhatsApp migration tools are my own set of utilities and documentation for moving WhatsApp databases from Windows Phone to Android, as well as a comprenhensive description of the 'forensics' involved in manipulating WhatsApp databases.
 
-There are methods explained all over the internet on how to get access to data in WP, check [this link](https://www.winwazzapmigrator.com/faq/windows-how-extract-messagesdb) for example.
+Previously known as [WAToAndroid](https://github.com/ferferga/WhatsAppMigrationTools/blob/1.0/README.md), I added one more tool and detailed instructions on how to successfully have a working setup after running my scripts (which I hope that will be really useful now, just after the dead of WhatsApp and WP)
 
-This script lacks the ability to encrypt the database, as I was unable to get the way WhatsApp encrypts their databases using a Google's account. This means that this script
-might be competely obsolete in the recent WhatsApp versions. Thus, this script is provided AS IS, there will be no further updates or support from my side, as it already served for my purposes
-and WhatsApp is changing their databases very quickly.
+My tools are different from others like [Whapa](https://github.com/B16f00t/whapa), as mine are designed to merge WP databases, which aren't covered in many other third-party tools.
 
-## Credits
+### A bit of background
 
-You are free to use this script and modify it as you want, but please, give credits if a function or a piece of my code in this software is used in any of your projects.
+(Read the original description before reading this [here](https://github.com/ferferga/WhatsAppMigrationTools/blob/1.0/README.md), for even a better *background* =P)
+
+Even though I said that this repository wouldn't receive any further updates, some other complications arose after publishing it. 
+
+The original script was written between the middle and near the end of 2018, as I was already looking into switching back to Android after some years with my Lumia 640 XL, and also wanted to have another useful project in hands for learning. 
+I tested restoring the resulting DB in an spare Android phone I got and it was a success back then.
+
+However, I got a new phone just at the end of January 2019. I tried my own script without success, as WhatsApp already changed the way the restore process works,
+not allowing local unencrypted backups to be restored. As I already had everything moved over to my new phone (confident that it would work), I was in a rush to get WhatsApp moved over as well. I discovered then ``WinWazzapMigrator``
+an app made by *Nicola Beghin* (who seems that had the same problem I had and wanted to solve it) that served me perfectly. As my script proved to be useless, around that date, I decided to upload it here for people who might find it useful anyway (although it was already probably useless for everyone, not just me).
+
+However, during this 2019 year, I discovered some issues with WinWazzapMigrator that really annoyed me:
+
+* There was a starred message that crashed the app everytime I opened the 'Starred messages' section of the app.
+* Messages quotes and mentions were completely away.
+* All the call logs were missing
+* I decided to not carry media over to the import, in order to save some space. However, even if I wouldn't be able to download them again (because WhatsApp couldn't locate them in the internal storage of my phone) I thought that WinWazzapMigrator would carry over the blurry thumbnails (example below) and placeholders for voice notes and all the media.
+
+<p align="center">
+  <img src="https://github.com/ferferga/WhatsAppMigrationTools/raw/master/images/blurred_media.jpg">
+</p>
+
+All these problems led me to many of my conversations lacking any sense. Also, I became aware afterwards that I could simply use a rooted Android system to access WhatsApp data. I started to investigate around this and I finally decided to write another tool to fix those quirks and merge both databases.
+
+## Project structure
+
+A brief description of the folders included in this repository:
+* apk: This folder contains ``WhatsApp 2.18.248`` apk's, as they are known to be the last version where WhatsApp allowed the restore of local encrypted/unencrypted backups. This might be useful if you are moving from WP to Android directly without using WinWazzapMigrator.
+
+* converter: This folder contains the old ``WindowsToAndroidWhatsApp`` (renamed to ``WPWhatsAppToAndroid`` + some tiny changes from last commit for simplifying the job to the ``merger.py`` script). This script will convert your Windows Phone databases to the old schema used in Android databases.
+
+* images: Just images for the documentation
+
+* merger: This folder contains the script that will merge your old WP database with an existing Android database (created by the WhatsApp app in Android).
+
+* samples: This folder contains sample databases of each type for those aiming to upgrade my script if they stop working (if that's the case, please, make a PR!) or for other research purposes.
+
+* LICENSE: Licensing information of this repository. This project is licensed under GPL3.
+
+* requirements.txt: Being Python scripts, my script requires one module that you need to install before. If you're not under Windows, run ``pip install -r requirements.txt`` in your system to install the required module.
+
+If you're a Windows user (or another OS user that needs bytecode Python for some reason), under the [Releases tab](https://github.com/ferferga/WhatsAppMigrationTools/releases) you will find pre-compiled Windows executables. Just extract them and run. No hassle.
+
+## What do I need?
+
+A clone of this repository made with ``git clone https://github.com/ferferga/WhatsAppMigrationTools`` and basic-intermediate computer knowledge (Please, if you think you get stuck during the process due to your knowledge, google a bit first, as the answer to your question might be out there already! That way, I would be able to help better and faster when needed)
+
+## Where do I start?
+
+Start first using the converter script called 'WPWhatsAppToAndroid', check over [here](https://github.com/ferferga/WhatsAppMigrationTools/blob/master/converter/README.md)!
+
+## Is this for me?
+
+This might be what you are looking for if:
+
+* **You did use WinWazzapMigrator to move from WP to Android**: This is what I did, and what was tested
+
+* **You moved to Android starting from scratch there a while ago, and now you want to bring back all your old WP messages to your current WhatsApp installation.**: This will likely be successful, although it wasn't tested.
+
+* **You still didn't move from WP and don't want to buy WinWazzapMigrator**: My process is much more difficult and less user-friendly than theirs, but it will likely succeed as well, although it isn't tested. I would say however that this has less chances of being successful than the 2nd case. But worth trying I suppose, you will learn a lot along the way :).
+
+If you see that any of the examples fits you, take a look anyway, as there are some things (like the *sorting database* part of the merger script) that can be helpful.
+
+# DISCLAIMER
+
+All of these tools and information provided here are intended for educational purposes and only in good faith. I don't encourage anyone, in any way, to try these methods to modify or extract databases of other people. Also, I **don't take any responsibility of what could happen when you use this information or tools I provide**
+
+Although I will try and do my best to help you in case something goes wrong, WhatsApp changes databases very quickly (even if they don't end up implementing many of the features they start to code) and I might not be able to tell you how to solve an specific problem (specially because it might be caused by WinWazzapMigrator, if you used it before, instead of my scripts). So, **ALWAYS KEEP BACKUPS OF ABSOLUTELY EVERYTHING!**
+
+# Credits
+
+You are free to use this script and modify it as you want, but please, give credits if it you use any part of this work in yours. Also, if you improved something, don't hesitate to make a Pull Request!. 
+If you're coming for a press, it's always great to get people from other sources coming to the repository and give their feedback on your original work, so please link to this as well if you make an article of this.
+
+Everything that you have here is a product of countless hours (across many months)for researching, testing and writing the documentation, so I appreciate all of this.
+
+Thank you very much!

apk/README.md

@@ -0,0 +1,16 @@
+# WhatsApp APK
+
+During all my research in ways in accessing crypted databases, I found
+some websites claiming that the latest version that supports restoring local backups with and without encryption (like the ones generated by the main 'WPWhatsAppToAndroid' script)
+is ``2.18.248``. 
+
+Here you can find all the APK variants from that version, directly downloaded from APKMirror. They will be useful for you if you want to try to restore the
+android database generated by my 'WPWhatsAppToAndroid' script (although I'm still unsure if it works). You shouldn't need them if you follow my new instructions, but here they are, just in case if useful for another backup or something else.
+
+As always with software from unkown sources around the internet, use it at your own risk and take the proper saftey measures. 
+I didn't checked, neither used them, so you've been warned.
+
+## Troubleshooting
+
+If you're from far away from the future, maybe, after installing it you will be welcomed with a message saying that the version you have installed is expired.
+Change the date of your device to a date near August of 2018 (the date when this version was released). After doing this, you should be able to make a backup and go to a newer version

apk/whatsapp_2.18.248_arm64-v8a-nodpi.apk

@@ -0,0 +1,46 @@
+# WPWhatsAppToAndroid
+
+This script will convert your existing Windows Phone's WhatsApp database to
+the Android's schema.
+
+## What do I need?
+
+You need two databases: **messages.db** and **calls.db**, which you can get
+using this [nice tutorial](https://www.winwazzapmigrator.com/faq/windows-how-extract-messagesdb) from WinWazzapMigrator.
+
+**calls.db** is not mentioned anywhere in the tutorial (as WinWazzapMigrator doesn't move them), however, it's located in **messages.db**'s directory, so you will spot it easily.
+
+## Before starting
+
+* **Windows users**: Download [here](https://github.com/ferferga/WhatsAppMigrationTools/releases) the latest available release.
+
+* **Unix users**: Clone this repository and install the requirements as specified in the main 'README.md' file.
+
+**ATTENTION!:** There are some part of the code designed to work only with Spain's phone numbers, so my contry code is used (+34). Go to lines **282**, **334** and **335** and change *34* with the phone code of your country. If you have contacts from other countries, you might need to do further changes to the code.
+
+Also, take a look at lines **342** and **344**. There, (I'm not sure, as I never had my hand on a non-spanish database) I suspect that the ``s`` letter stands for 'Spain'. 
+
+Open first your database with [SQLite Database Browser](https://sqlitebrowser.org/) and check the 'Jid' column in 'JidInfo' table to see how your contacts jids are stored.
+Most of the records in that table should be something like [email protected] (being **X** your country code, **Y** the phone number of your contact, and **z**). Ignore the rest, only pay attention to those that ends on ``@z.whatsapp.net``
+
+Replace the ``s`` letter in the lines mentioned above if your **z** is different than ``s``.
+
+Any PR to make this script easier to 'globalize' is appreciated!
+
+(Of course, changing the lines in the code means that Windows users will need to run the script from its source, with Pyhton installed in their systems)
+
+## Instructions
+
+The process is really straightforward, everything is explained inside the script, so just follow the instructions.
+
+After finishing, **don't remove any of the files**, you will need them later.
+
+## Done! What's the next step?
+
+Go over [here](https://github.com/ferferga/WhatsAppMigrationTools/blob/master/merger/README.md)!
+
+## Issues
+
+If facing any issues, publish the exception code received. For seeing it, just reproduce what you were doing again in a cmd or bash terminal, executing the script from there (instead of double clicking on it in the file browser), as the script might simply crash.
+
+**IF YOU FACE AN ERROR, YOU ARE SUGGESTED TO NOT USE THE RESULTING *MSGSTORE.DB* DATABASE IN ANY OF THE FOLLOWING STEPS, AS YOU MIGHT END UP WITH AN INCOMPLETE OR INCOHERENT DATABASE**