(photo: Antony Hell)

Sometimes you have to call it quits and admit your phone just isn't working right, and no number of help group suggestions is going to fix it. But, maybe, a factory reset will. Maybe you should just start over.

And yet, you don't really want to start over. You want your email, and your pictures. You want your apps, even if you have to reinstall them. You definitely do not want to wipe your phone only to discover you deleted something super-critical.

This guide can't guarantee you won't lose something important. There are too many different apps, settings, and places to hide. But it should give you a better chance, maybe make you think of something you hadn't before.

These instructions are for a Windows user. Sorry, Mac people. Also, my phone is a Nexus 5 running Android 7 (Nougat). Your settings and dialogs may have different labels.

Basically

1. Get your account passwords ready
2. Clean up your installed app lists
3. Backup all your data
4. Factory Reset
5. Setup
6. Test

Preparation

Set aside plenty of time for this. Make it easy on yourself by having:

• Your phone's power adapter and cable.
• A USB cable to attach to your computer.
• WiFi access, otherwise the data usage will eat you up.
• Fully charged your phone.

WiFi and Cell Service

You shouldn't have to do anything to restore your cell service after the factory reset, That information is tied to your SIM card, or otherwise stored in a non-volatile way. But, I can't guarantee you won't need to enter some kind of information required by your carrier.

If you have WiFi, you'll need your WiFi password, and you'll need to know your wireless network name.

Apps

Cleanup

By default, Google remembers every application you ever installed from the Play store, even if you uninstall it. After factory resetting, and entering your Google account, there's an option to restore all your installed apps. You may want to do that, or you may want to reinstall from scratch. I like doing the latter, because

1. I only install what I really need, and
2. I don't know if a previously installed app is causing my problems.

So, I want my application history to only show what's actually on my phone. Thankfully, the latest versions of the Play store make this easier that before. To clean up your apps:

1. Open the Play Store app.
   

2. Switch to the Library tab. This is all the applications you've installed in the past, but are not currently installed.

   

3. Tap the X to delete applications that you don't plan on installing again. Be ruthless here. If you can delete them all, do it, because after the factory reset this tab should suddenly contain the applications you currently have installed. The goal is not to get confused when you manually reinstall apps.

Important

You cannot do this cleanup from the Google Play web site. You can only do it from your phone. So do it before you reset your phone.

Review

Now open the Installed tab.

This is a good time to review your applications and ask a few questions:

1. Do I know this app's user/password?
2. Does this app store data that I want to keep on my phone?
3. Does the app have settings that I don't want to forget about, that were a pain to get working?

Other Apps

Did you install any apps from sources other than the Play Store? One likely culprit: Amazon, specifically their Amazon Underground and Music apps.

Passwords

Do you have applications that require a password, but you never have to enter one? Make sure you create a document with those passwords and usernames. Double check what's in your document by logging off from the app and reentering the app's username and password from the document. If you have a password manager, make sure you know and test the master password. Here are some apps might be auto-logging you in.

• Your Google account. You probably never enter your Google password on your phone.
• Other accounts. Look in Settings > Accounts
• Microsoft Office apps (Word, Excel, etc)
• Facebook, Twitter, Snapchat...all social media
• Amazon Kindle and shopping
• Netflix, Hulu, etc.
• Your banking app

The good news is, you can almost always recover your password by going to the application's web site.

Phone Backup

There are backup programs you can use to backup your entire phone, and then restore it later. Unfortunately, I can't recommend anything because never use those. Sorry. But here's a recent web article about backup apps.

10 best Android backup apps and other ways to backup Android!

The advantage to a full backup app is, if something goes wrong, you should be able to put your phone back the way it was. But don't rely on that. Take other steps, below, to protect your data.

App Data

Your phone's apps aren't your life. Your primary goal is to be able to recreate your data.

Your data is your life.

First, here's a quick tutorial on connecting your phone to your computer and seeing its files. Most of you know how to do this, but it's worth documenting here.

1. Plug your phone into your computer using a USB cable.
2. Your phone should be recognized. You might be prompted how to use the files. Choose "Transfer files" Don't choose to just transfer photos (PTP or MTP). We want access to as many files as possible.
3. You may need to enable data transfer from your phone. Swipe down and open "USB charging - Tap for more options", then choose Transfer Files
          
4. On your PC, open the File Explorer, expand This PC, and find your phone's name.
   
5. Select that folder, open Internal shared storage, and all you'll have access to your phone's files (most of them).
   

You can recreate your data in these ways:

1. Keeping it stored in the cloud. That's the way it works for Google, Facebook and Twitter. In general, if you can access your account and data from both the web and your phone, the data's in the cloud. These are easy.
2. Exporting it in a file from the phone to your computer, and Importing that file later. Also called Backup and Restore.
3. Copying it from the phone's application or document folders to your computer, then copying it back later.
4. Recreating it by hand. You're trying to avoid this.

Cloud

Many applications store your data in places that are hard to get to. Often they store the data in the cloud, which is great because all you have to do is log in and it'll be there.

Backup/Restore or Export/Import

Others store the data on your phone. For example, I have a nice time tracking application, Timesheet - Time Tracker, that stores backups on the phone. It has a paid cloud sync feature, but I don't use it. Checking the settings, I see that I can also backup and restore files from my Google Drive, so I do that, verify the backup file is there, and test restoring the backup.

Backups are nice. But it's the restore that counts. Test your backups and exports.

If you're prompted where to export to, I recommend two places: your Google Drive in a folder named Phone, and/or your phone's Documents folder. In either case, remember to name the file so you'll know what it's for later, or put it in a well-named folder.

If you export to your phone, you'll see later how to copy those files to your computer.

Copying

A thankfully-very-few applications store their data but don't provide a way to export it. That's a hard problem. If you're not very phone/tech savvy, you're probably stuck recreating it by hand. If you do know your way around your phone's file system innards, then try a couple of things:

1. Try to find the data in the Android folder, either in the data or media folder. Unfortunately, the data files are sometimes. Windows Search sometimes works, if you look for known file names.
2. Advanced users can install and use an Android file explorer to try to access hidden files, then copy those files to a standard phone folder such as Documents.

Personal Data

A lot of your application data is personal data, but there's a big chunk of stuff that might be on your phone and nowhere else. That chunk is likely to be:

• Pictures
• Downloads, such as PDFs or videos
• Anything you copied or moved to your phone from your computer

Copying this data is pretty easy.

1. Connect your phone to your computer and open its files (see above).
2. Create a folder on your Desktop or Documents called Phone and open it.
3. Copy everything from your phone to that folder.

DO THIS!

This may take a while. In my case, it was only about two minutes. But if it takes longer, that's OK. You'll have captured everything that you possibly can. Do this after you've exported app data as shown above.

Looking at the folders, you might wonder where your camera pictures are. They're in the DCIM folder.

Breathe and Continue

At this point, you should have captured any data that will be lost from your phone. Now, stop. Walk away for five minutes. Come back. And review. Really, do this, because you'll probably think of something you missed. That five minute break may save you hours later.

Factory Reset

This is pretty easy.

1. On your phone, open Settings > Backup & reset > Factory data reset
2. You'll get a nice warning about losing data, and which accounts you're signed in to. Do you have usernames and passwords to these?
      
3. When you're ready, choose Reset Phone.

Android will wipe your phone and reinstall the current factory version of the operating system.

Factory reset does not mean "like it was when I bought the phone." You'll get the latest version of the Android operating system and updates that you had installed.

Setup

After the reset, you'll go through the initial setup. This will include:

1. Reconnecting to your cell carrier, which should happen automatically.
2. Connecting to your WiFi, if you're using it.
3. Entering your Google account information. Be careful here. You'll be prompted to reinstall all your apps. Think carefully whether you want to do this or install them manually.

After your account is set up, you'll go through the tedious process of reinstalling your applications from the Play Store. But I think you'll find that you can prioritize to just the apps you need to survive and be productive (especially if you use your phone for work).

The biggest hassle is, frankly, all the little settings you didn't know you had, getting your phone working just the way you like it.

Don't Rush

Are you doing a factory reset because of phone trouble? If so, don't rush to reinstall your apps. After the initial setup, before reinstalling anything, check if the problem has gone away? If so, great!

Now install the apps one at a time, and check for that problem after each installation. This might take hours, or days, but you might find an offending application and save yourself lots of headaches later.

You might just save your mind.

This series simulates a native application accessing a protected Web API resource, using OAuth2 via IdentityServer3. It demonstrates using Proof Key for Code Exchange (PKCE), and is in four parts:

1. Build a simple authorization server, consumed by native application.
2. Build a protected resource.
3. Persist server configuration to database.
4. Persist user data to database using Microsoft.Identity and SQL Server.

Important
This series does not create an OpenID Connect (OIDC) server. It is OAuth-only, since the PKCE specification doesn't require OIDC.

Where We're At

We're persisting IdentityServer configuration data, but still using in-memory users. IdentityServer can be extended to validate against an identity store, and supports Microsoft's AspNet.Identity via Entity Framework on SQL Server.

Important IdentityServer is not intended or used to add, edit or delete users. They make that clear. The server's job is only to verify identity. In this sample, there's a klunky method for populating the user database, and there are no API methods for registering a user. The API could be added using controller code similar to what's in the default ASP.NET Web API templates.

AspNetIdentity

References
IdentityServer3.AspNetIdentity Sample
Identity Server 3 using ASP.NET Identity
OAuth2 Walkthrough using Identity Server and ASP.NET

Install these packages.

Install-Package IdentityServer3.AspNetIdentity -Project AuthAndApi
Install-Package Microsoft.AspNet.Identity.EntityFramework -Version 2.2.1 -Project AuthAndApi

Add a new connection string to web.config, for the AspNet Identity database.

    <add name="IdUsersDb" connectionString="Server=(localDb)\MSSQLLocalDb;AttachDbFilename=|DataDirectory|IdUsers.mdb;Database=IdUsers-ncacpkce;Integrated Security=true;" providerName="System.Data.SqlClient" />

Update Factory.cs

//Add these usings
using Microsoft.AspNet.Identity.EntityFramework;
using Microsoft.AspNet.Identity;
using IdentityServer3.AspNetIdentity;
using IdentityServer3.Core.Services;
using IdentityServer3.Core.Services.InMemory;

Change the Configure method, adding code to persist the users, and a new method to populate the database.

        public static IdentityServerServiceFactory Configure(string optionsConnectionString, string usersConnectionString)
        {
            //Configure persisting IdentityServer operational services
            //The database will be created if needed. The default schema is being used.
            var efConfig = new EntityFrameworkServiceOptions
            {
                ConnectionString = optionsConnectionString
            };

            // these two calls just pre-populate the test DB from the in-memory config.
            // clf note: there's probably a better way to do this for a production app.
            ConfigureClients(Clients.Get(), efConfig);
            ConfigureScopes(Scopes.Get(), efConfig);

            var factory = new IdentityServerServiceFactory();
            factory.RegisterConfigurationServices(efConfig);
            factory.RegisterOperationalServices(efConfig);
            factory.ConfigureClientStoreCache();
            factory.ConfigureScopeStoreCache();
            //persist users
            //populate the database. This creates the database, if needed.
            ConfigureUsers(Users.Get(), usersConnectionString);
            //configure IdentityServer to use the database
            factory.UserService = new Registration<IUserService>(UserServiceFactory.Create(usersConnectionString));

            //configure cleanup for expired data
            var cleanup = new TokenCleanup(efConfig); //by default every 60 seconds

            return factory;
        }

        public static void ConfigureUsers(IEnumerable<InMemoryUser> users, string connectionString)
        {
            //create the Identity UserManager
            var context = new IdentityDbContext(connectionString);
            var userStore = new UserStore<IdentityUser>(context);
            var userManager = new UserManager<IdentityUser>(userStore);
            foreach (var user in users)
            {
                //no error checking!
                //only add if doesn't exist
                var test = userManager.FindByName(user.Username);
                if (test != null) { continue; }
                userManager.Create(new IdentityUser(user.Username), user.Password);
                test = userManager.FindByName(user.Username);
                foreach (var claim in user.Claims)
                {
                    if (claim.Type.ToLower() == "email") { test.Email = claim.Value; }
                    userManager.AddClaim(test.Id, claim);
                }
                userManager.Update(test);
            }
        }

And finally, add a new factory class, which is used in the line factory.UserServce =.

   public static class UserServiceFactory
    {
        public static AspNetIdentityUserService<IdentityUser, string> Create(string connectionString)
        {
            var context = new IdentityDbContext(connectionString);
            var userStore = new UserStore<IdentityUser>(context);
            var userManager = new UserManager<IdentityUser>(userStore);
            return new AspNetIdentityUserService<IdentityUser, string>(userManager);
        }
    }

That's it! The Startup class doesn't change. Running the solution creates the new database, and the authentication works as before.

Wrap Up

In this series, I demonstrated authorizing a mobile application in a secure way. This was necessarily a simple sample. Some natural next steps and questions to answer would be:

1. How to customize the IdentityServer login and authorization screens
2. Configure for OpenID Connect3
3. Show using Google authentication

This series simulates a native application accessing a protected Web API resource, using OAuth2 via IdentityServer3. It demonstrates using Proof Key for Code Exchange (PKCE), and is in four parts:

1. Build a simple authorization server, consumed by native application.
2. Build a protected resource.
3. Persist server configuration to database.
4. Persist user data to database using Microsoft.Identity and SQL Server.

Important
This series does not create an OpenID Connect (OIDC) server. It is OAuth-only, since the PKCE specification doesn't require OIDC.

Where We're At

Until now, we've been using in-memory configuration data (clients and scopes). But we'd like a way to persist the configuration to a database.

IdentityServer and Entity Framework

References

Entity Framework support for Clients, Scopes, and Operational Data
Operational Data
Clients and Scopes
Caching results for client, scope, and user stores

Add these packages.

Install-Package IdentityServer3.EntityFramework -Project AuthAndApi

Add the App_Data folder. This is required if you want to use the connection string below.

Add a connection string for the configuration database. The sample uses SQL localDb 2016.

<connectionStrings>
    <add name="IdConfigDb" 
         connectionString="Server=(localDb)\MSSQLLocalDb;AttachDbFilename=|DataDirectory|IdConfig.mdb;Database=IdConfig;Integrated Security=true;" 
         providerName="System.Data.SqlClient" />
  </connectionStrings>

Add an IdentityServerConfiguration folder, and a Factory.cs class. The factory class takes care of setting the configuration database source, and also initially populating the database using our in-memory client and scope collections.

Note
This sample doesn't provide a way to edit configuration data. There is a NuGet package that adds web-based configuration administration to a project. I haven't tried it, but here's the link. IdentityServer3.Admin

Factory.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
//Added
using IdentityServer3.Core.Configuration;
using IdentityServer3.EntityFramework;
using IdentityServer3.Core.Models;
using AuthAndApi.IdOptions;
using AuthAndApi.IdUsers;

namespace AuthAndApi.IdentityServerConfiguration
{
    public class Factory
    {
        public static IdentityServerServiceFactory Configure(string optionsConnectionString)
        {
            //Configure persisting IdentityServer operational services
            //The database will be created if needed. The default schema is being used.
            var efConfig = new EntityFrameworkServiceOptions
            {
                ConnectionString = optionsConnectionString
            };

            // these two calls just pre-populate the test DB from the in-memory config.
            // clf note: there's probably a better way to do this for a production app.
            ConfigureClients(Clients.Get(), efConfig);
            ConfigureScopes(Scopes.Get(), efConfig);

            var factory = new IdentityServerServiceFactory();
            factory.RegisterConfigurationServices(efConfig);
            factory.RegisterOperationalServices(efConfig);
            factory.ConfigureClientStoreCache();
            factory.ConfigureScopeStoreCache();
            //Still using in-memory users for now
            factory.UseInMemoryUsers(Users.Get());

            return factory;
        }

        public static void ConfigureClients(IEnumerable<Client> clients, EntityFrameworkServiceOptions options)
        {
            using (var db = new ClientConfigurationDbContext(options.ConnectionString, options.Schema))
            {
                if (!db.Clients.Any())
                {
                    foreach (var c in clients)
                    {
                        var e = c.ToEntity();
                        db.Clients.Add(e);
                    }
                    db.SaveChanges();
                }
            }
        }

        public static void ConfigureScopes(IEnumerable<Scope> scopes, EntityFrameworkServiceOptions options)
        {
            using (var db = new ScopeConfigurationDbContext(options.ConnectionString, options.Schema))
            {
                if (!db.Scopes.Any())
                {
                    foreach (var s in scopes)
                    {
                        var e = s.ToEntity();
                        db.Scopes.Add(e);
                    }
                    db.SaveChanges();
                }
            }
        }
    }
}

Update Startup.cs, replacing the previous IdentityServer configuration that explicitly configured the Factory with in-memory data, with the new call to Factory.Configure.

            //Configure IdentityServer for issuing authentication tokens
            var options = new IdentityServerOptions
            {
                Factory = Factory.Configure("IdConfigDb"),
                //SSL MUST be used in production
#if DEBUG
                RequireSsl = false
#endif
            };
            app.UseIdentityServer(options);

ASIDE!

Running the app at this point, I got an error in Startup.cs that I was missing a dependency.

 

When I installed the package IdentityServer3.EntityFramework, Newtonsoft.Json version 8 was uninstalled, because the package requires version 9 or greater. The package installer output confirms this.

Successfully uninstalled 'Newtonsoft.Json.8.0.3' from AuthAndApi
Successfully installed 'Newtonsoft.Json 9.0.1' to AuthAndApi

No package requires version 8 or lower, so what gives? For some reason, the solution's web.config assembly binding section wasn't updated properly. It still had this:

     <dependentAssembly>
       <assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" culture="neutral" />
       <bindingRedirect oldVersion="0.0.0.0-8.0.0.0" newVersion="8.0.0.0" />
     </dependentAssembly>

You could manually change the max version number to 9.0.0.0, reinstall the Newtonsoft package which will do it for you.

Update-Package -Reinstall Newtonsoft.Json -Project AuthAndApi

When I run the application, it behaves exactly as before. Checking in the App_Data folder, I see my database was created.

One difference, though. After the first time running the app, I no longer was prompted to authorize the scope. This might be related to these factory settings:

factory.ConfigureClientStoreCache();
factory.ConfigureScopeStoreCache();

Wrap Up

Our server's configuration is now being persisted to a database. The only thing (for this sample!) left is to persist the user's credentials and claims.