Put simply, it's like a window manager for your terminal. No longer will you be burdened by 10 iterm tabs as you start up your local project, ssh to your server, keep webpack running, and download in the background.

Suddenly, as you ssh into your remote machine you have all the window management capabilities you have locally. Much like my opinions on vim, it doesn't take a lot for tmux to be really useful.

Get started?

Install, then run tmux. Well done, most of the way there. To run comamnds in tmux you use a pair of keyboard shortcuts. The default is Ctrl + B. This is often called the Leader. So you'll see things like "Do this thing by Leader + %". That means press Ctrl + B and then press %.

Window Pane Management

To rekindle your love of tabs use Leader + c (C for create). Congrats, on the bottom bar you'll see a new "Window". To close this window, you can simply exit your terminal. There is a keystroke for it, but i've never used it. Want to switch between windows? Just use Leader + <number> where number is the window index.

Splitting the window is great, to do this you want Leader + % or Leader + ". Want to know how to remember them? Me too. I've found this easier to learn by attrition. " creates a horizontal switch. % makes a vertical one.

Tmux is now useful

At this point in tmux we haven't covered plugins, tmuxconf, getting bloody copy and paste to work nicely on a mac. But you don't have to know a lot about tmux for it to be useful. With the above you're already able to jump onto your remote AWS machine and spin up a series of terminals like a pro. No longer creating 5 different ssh sessions, but just one and multiplexing the terminal instead.

Much like as mentioned in the vim post, the consistency that can be delivered to a developer by the use of tools like tmux and vim can mean that there's no discernable difference between you working on your local machine and when you're running something remotely.

Well, you don't have to know a lot about vim for it to be a really useful tool. It's a great way to have a consistent editing experience across servers, IDEs, or even Chrome. With very basic knowledge it also begins to become something that'll make a lot of your work faster, and easier.

So, this aims to provide the precise minumum that you can know about vim and still find it useful. This means, i'll leave out things like visual mode, macros, and plugins - because fundamentally, you don't have to know a lot about them for vim to be useful.

Step 1 - You edit, or give actions

When programming, you'll almost definitely spend less time typing in characters compared to time doing literally anything else. So thus, when you are attempting to type in letters ensure you're in INSERT mode. You get into insert mode by typing i. If i just appeared in your document, congrats, you were already in INSERT mode.

While here, do all the typing of characters that you need. When you're done. Exit INSERT mode with the escape key.

In essence, to make Vim useful, get in the habit of being in insert mode when you actually want to add characters to your screen. The moment you're no longer doing that. Press escape.

Step 2 - Move around

When you're not in insert mode, you can give commands. You might want to use them to move. The most basic forms of movement is up, down, left right. Everything else is optional. It's just HJKL = Left, Up, Down, Right.

Step 3 - Save and exit

Because you're not inserting text, you're of course in COMMAND mode. In this mode press Shift + ; to begin your command. All you have to know to be useful is w, q, and !.

:w<Enter> is write

:wq<Enter> is write and quit

:q<Enter> is quit

:q!<Enter> is quit!!! I really mean it, don't ask about unsaved changes

At this point, Vim is already useful

Congrats. You now know enough to have a consistent editor no matter what platform you're on.

From here, every addition is a choice and thus it's worth avoiding beating youself up about the fact that you don't know every keystroke on that vim cheatsheet you saw. Instead celebrate the utility you already have.

Choices I made

In the 10 years i've used vim, here's a list of things i've never really gotten into

• A super customised ~/.vimrc file. The vast majority of the time, i don't even use a custom one though if you want one, just Google.
• Plugins
• Having multiple buffers open at the same time
• Trying to make Vim an IDE
• Macros
• File management within Vim itself

Still, any time that I install an IDE the first thing i do is install vim, i use it daily at least on my local media server, raspberry pi, or work laptop.

Here's the things i decided to learn about on top of it

Fast Movement

• gg top of File
• G bottom of file
• :12 go to line 12
• 23j go down 23 times. (Can do this sort of things with tonnes of commands)
• /searchString find "searchString" in your doc
• * find the next time the word under the cursor is mentioned
• n give me the next one
• p previous one thanks
• 0 start of line

Fast edits

• dd delete line
• C change from my cursor to the end of line
• ciw Change in word (delete the word and start INSERT mode)
• di{ delete all text within braces
• ci) delete all text within parens and start INSERT mode
• A append to the end of the line

This works for me. It only scratches the surface of what it can do. But that's ok. Instead of trying to be a master, try lean in to the fact that you don't have to know a lot about something for it to be useful.

In the grand scheme of things, I’m still quite new to my career in software engineering. I’ve been working in the field since 2013, at four separate places during that time. Three of those places had some form of code-review as part of standard process. One, a bank that performed code review as part of the process after code was on SVN mainline. Two, where code-review was a strict process where discussions could go on for hours before your code ever hit the master branch.

When you join these places, part of the engineering onboarding is usually talking about how one goes through the process of PRs (Pull Requests). In particular it is mentioned that PRs really should be merged in 1-2 days. Each time though, the actual time to merge a PR is higher. Sometimes lots higher.

What follows is a personal opinion on what you, and I can do to help lower this. This method is something that I’ve used in the past to great success, and you’re able to pick up as just an individual, this doesn’t require the whole company to try it all at once.

Lets start with a bit of why I think these issues arise.

What’s hard about a PR

For a normally sized PR, there are a few things that are difficult when reviewing

• Context
• Time
• Latency

Context: When work across teams, or even in large teams you're often assigning review to someone who knows the code well, but might not be fully aware of all the work you're doing. To get around this, as a good PR owner, you of course wax lyrical in the PR description about what your PR is, and why it’s doing it. As a reviewer though, it’s a bit of a lottery if that’s enough to go on, or whether it answers the kind of questions that you need answered.

Time: As a reviewer, often you’ll just get hit with a PR in your inbox without much warning. Maybe you knew it was coming, but there’s often cases where you just suddenly have a fresh piece of work on your plate that’s just arrived without so much as a ping on Slack. Now your week just got effectively shorter for you to actually perform any work that you wanted to do.

Latency: This affects both sides of the review. It’s hard to ask questions, and get responses fast. If you ask a question one day, it’s very possible that by the next day the other person on the review has forgotten about them, and will need to nudge them for an answer. Maybe you’re answering any questions asked on your PR in seconds, but your reviewers never seem to get back to you as fast. Perhaps because they’ve expected it to take a while to receive a response, and thus have moved on to the next review. For every question that goes back and forth in a PR you can lose somewhere between minutes, and days of time.

How can you fix that?

With Pair code review.

In my opinion it’s as simple as that for a vast amount of cases. In a perfect world here’s my steps for a quality, and fast code review:

Before you start work on code, you should have an understanding about who you’ll be needing as a reviewer, notify then that you’ll be interested in performing a pair code review. Just to be polite about what you’re expecting from the reviewer.

Now, complete the code (the easy part).

Put up the review. The review quality should be just as good as always. This technique does not allow you to have a lower quality description, or lower quality clarifying comments throughout the PR.

Here's where things take a small turn.

Send a calendar invite to your reviewer(s). When doing so, I follow these rules

1. Your PR should be sized that this should take 30m - 1h max. If it’s a larger PR, break it down. No excuses. This is especially important for this technique.
2. Your calendar invite should respect the reviewers time, make sure you’ve not just turned their day into a crap meeting-fest.
3. You should aggressively give back time. Indicate this on the invite. If the pair-review only takes 15m of the 30m, end the meeting straight away and relish. Nothing is worse than someone stretching a meeting just because it finishes 10 minutes from now.
4. When it's time for the start of the meeting, make sure you’ve got a good place to go to. If you’ve invited more than one person, grab a room with a TV, otherwise go to the reviewers desk.

Running the review as a reviewee

Your job is to sit near/next to your reviewer and attempt to help with any and all context. You’re there to answer the "why did you do this" kind of questions. You should be taking notes, for all the changes that are desired.

Often i’ll have the review open on my own laptop, and write down what the reviewer tells me to change as comments on the PR like they’d done it.

Make sure to ask any clarifying questions straight away. Get that latency down to seconds rather that hours or days.

Come prepared with justifications for what you’re doing. It should be rare that you answer a question with ‘i don’t know'. Remember that you’re asking for the gift of your reviewers time, respect that by being prepared.

If a large change is required and/or its clear the review is going to be rejected, clarify how the solution should be rearchitected. Then, stop the meeting, and give back the reviewers their time. Alternatively, if you’d like this time can be used for some short pair programming to make the next session more likely to succeed.

As a reviewer

If possible, it’s good to have spent the 5-10 minutes before the calendar invite familiarising yourself with what’s there. This doesn’t have to be time reviewing, but should help you know if you’re the right person for the job, and have your brain running in the background about things you should check. Often the best time to do this is as close to when you received the invite as possible. This will allow you to respond fast if you're not the right person for the job.

When in the meeting, you don’t have to constantly talk, being silent is expected and totally ok. The reviewee is there to benefit your ability to review, but will be getting a lot of of this. You don't need to feel pressured to make conversation, or constantly update what you're thinking. You should feel free to review as you usually would. The only change is to verbalise your actionable comments, and ask questions straight away rather than on the PR.

Be clear at the end of the meeting what actions you’re expecting from the reviewer, and if they need to either book more time for a second review or, if you’re happy for them to merge once the changes you’ve asked for are applied.

For subsequent reviews, sometimes this whole process isn’t needed. Chances are your views have become so aligned that the number of questions, and thus latency is going to go down quite a lot.

Now all together, Merge, then celebrate.

What you get

This seems like a complex series of steps but overall boils down to

• Prepare your PR
• Review together

In my experience i’ve seen this consistently bring down PRs from up to 1-2 weeks to 2-3 days.

One thing to note is that this may feel like you’re losing time as a reviewer, as now you’ve suddenly got 10 calendar invites this week. My argument against that is that this approach should cut way down on your actual time spent per PR. There’s next to no context switching as questions are fielded back and forth without delay, thereby saving time. Your time spent writing questions and understanding answers is reduced to how quickly you can have a conversation.

As a reviewee, if you’re expecting to start a project that is going to require a lot of reviews from a particular person, do the courteous thing, and check in with them at the start. Make sure they have the bandwidth for this kind of thing. Perhaps you’ll need to help let their team know that you’re going to have to ‘steal’ them for a few hours a week as your project goes on.

Overall, feel free to do this, feel free to not. This is just a personal thing that I’ve found successful in the past, and would be happy to do in the future. This approach doesn’t need buy in from any more than you and your reviewer(s), so feel free to just give it a go one day if you feel that your PRs are taking a little longer than you’d like.

The worst part about that? It was all me. It was my fault. It was in my head. I knew that much years ago, but had zero ideas how to fix it. Dr Google only gets you so far.

To spoil a paragraph below, I kept doing something that's known as hyper-monitoring. I've only recently discovered the name for it, but it's basically something that occurs when you get stressed about sleeping. It's one of those annoying repetitive loops.

I would be concerned about my sleep before I went to bed
I'd try make everything perfect to sleep. Dark room, comfy mattress, sheets, silent
I'd not be able to sleep for an hour.
I'd constantly question why! Was I too hot? Too cold? Should I roll over? Lets try it, what i'm currently doing obviously isn't working?!

The questioning meant that my brain was constantly active, getting more and more frustrated as I was so tired yet everything I was doing wasn't letting me sleep
Thus, I'd not sleep, therefore making my concern higher the next night as I desperately needed to catch up
Therefore the next night i'd question faster, and be monitoring more things (temp, airflow, sound, etc)
It's a massive pain to have something that is so 'simple' be stuffed up and not know why. Kids can do it, some people can do it without even trying, it feels really lame to be shit at sleep. It affected me pretty consistently though, sometimes having to take days off work just based on ridiculous levels of tiredness.

Over the years I went to doctors, I'd heard everything you can hear about sleep hygiene. If you know about that, it is a list of 'good things' that you should do to have better sleep. Things such as not using your phone in bed, avoiding blue light, etc. One of the worst parts about moving to Sydney was that I was unable to actually have that any more. In a studio apartment, there's no avoiding lights from things in your house, no ability to separate your 'work space' and your 'sleep space'.

In the past, i'd been prescribed medication that helped for short times, but never underestimate the ability for your mind to defeat that, and I was soon back to bad sleep.

Finally, late last year I finally decided to specifically ask for a referral to a sleep psychologist. As of last week, i've finished my sessions and just wanted to provide details as I feel like sleep is something so many people have problems with but not many fix.

What do you do?

So, I went to see an absolutely fantastic sleep physiologist. Quickly it was reassuring to find out that what I was dealing with wasn't much when it comes to grand scale of sleep issues. At the start though, I was skeptical, I really didn't believe that I could think my way out of a problem. I was worried that a psychologist would provide wishy-washy techniques that were based on feelings rather than fact.

Oh damn I was wrong.

Over the next 6 sessions were some of the most evidence based hours I'd had the pleasure to experience.

First, we identified what I was doing. Even that was a relief. I remember calling my wife after the first session excited "She knows exactly what I do! That's so exciting! I'll be boring!". Boring is good. Boring is easy to fix. In general, what I had was a flight-or-fight response to going to bed. It sounds ludicrous, but when you spend hours just laying awake, bed becomes negatively connotated.

Second, and most exciting in some ways, was we started being very evidence driven. This was kind of weird, as I expected a lot of laying back on a couch with "how did that make you feel" kind of questions. Instead I had 2 worksheets I constantly filled out.

What time did you go to sleep?
What time did you wake up?
How did you feel your sleep was?
How awake did you feel during the day?
One of the first "facts" that surprised me was how low the correlation is between length of sleep and you actually feeling awake that day. Day tiredness and sleep quality are semi-unrelated, something Dr. Clancy would stress. This was great to find out weirdly enough. It reduced the stress of sleeping on an individual night as you're less worried about the next day definitely being terrible (still might be though).

Other things we did were techniques around CBT (Cognitive behaviour therapy), all of this with a large backing in research. I was able to collect evidence based responses for concerns that my brain would blow out of scale. eg "I didn't sleep at all night, my next day will be zero fun" → counter that with my birthday last year where I didn't sleep a single minute, but still had a freaking awesome time! Just reminding myself of this evidence made a bad sleep less powerful to affect my next day, and in turn became less scary, therefore letting me sleep easier.

It was also exceptionally useful to just have a really solid understanding of what's going on in your body around sleep. As someone who values evidence based stuff, everything was framed from the science of what's behind sleep (if you're interested, there's a great book by Matt Walker called "Why we sleep").

Overall

After close to 6 months, I'm way better than I was at the start. I still have nights where I don't sleep, or sleep very little, but they don't stress me any more. I accept them and don't get terribly stressed, it ends at one bad night rather than two or three weeks of awful times. This is incredibly freeing. Bed isn't inherently difficult any more. On top of that I now feel like I have the evidence based tools that will help me in the future when I kick back into a bad sleep funk for a little while.

Finally, I really would encourage everyone treat sleep like any other ailment. If you broke your leg you'd go to the doctor. If you've broken your sleep, it's one and the same. If it's your head that's stuffing you up, then I hope that knowing that it can be such a successful treatment will help justify going to get the help needed as well. I just wish I'd done it years ago.

A private method generally means that only the class that defined the method may call it. That said, Java does give you tools to poke holes in it from the start. Let's take the following as a starting point.

class Privacy {
   
    private Privacy() {
        System.out.println("Hi there");
    }
}

Ths post aims to answer "how can we get the JVM to call the private constructor?"

Method One: Reflection

Using the reflection APIs this is simple.

public class Reflect {
    public static void main(String... args) throws Exception {
        Constructor<Privacy> constructor = (Constructor<Privacy>) Privacy.class.getDeclaredConstructors()[0];
        constructor.setAccessible(true);
        constructor.newInstance();
    }
}

Lo and behold, the class is available and the instance is ready.

This is a feature of language, and can be guarded against if you don't want code to be using the reflection APIs. The following bans all reflection.

System.setSecurityManager(new SecurityManager() {
            @Override
            public void checkPermission(Permission perm) {
                if (perm instanceof ReflectPermission) {
                    throw new SecurityException();
                }
            }
        });

Method 2: Using inner classes

How else can we do it? Well, if you have access to the source code (and assuming we can't change private to public) then you can add an inner class.

class Outer {
    
    private Outer() {
        System.out.println("Hi there");
    }

    public static class Inner {
        public void goForIt() {
            new Outer();
        }
    }
}

In this case, we're very much able to run new Outer.Inner() and cause the JVM to output the message. This structure may be familiar to you if you utilise a builder pattern. Often, a builder's build() method will call straight back to the private constructor of the enclosing class.

What about a third way?

Let's assume for a moment that

• You're unable to use reflection
• You can't change any of the original source code.
• The class containing our private method we want to access has the following
  • An inner class
  • That inner class accesses the private member we desire (constructor/member)

Given that, let's start by taking a look at our example. This is the code that we want to try call the private constructor on. You can surely imagine that we very much would like to create a bank account with more than $100 starting balance.

class BankAccount {
    
    private BankAccount(int amount) {
        System.out.println("Bank balance: " + amount);
    }

    public static class BankAccountBuilder {
        public void balance(int i) {
            if (i > 100) throw new IllegalArgumentException("$100, yeah right");
            new BankAccount(i);
        }
    }
}

That is the same structure as we've just had. There's an inner class that calls the private constructor. This time though, we're not going to go through the BankAccountBuilder, we instead want to create our own instance without using the validation that's performed in the Builder.

These examples take places on the following machine. The overall method has been tested on the Oracle JDK, eclipse compiler, and the Open JDK. Though what you see in the rest of these sections are specific to Oracle, and will require slight adjustments for the other compilers.

Macbook pro 2015 - High Sierra
Oracle java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)

How does the inner class work?

To get started with this investigation, we can look at what's actually going on in the inner class. In the simplest case, let's take a quick peek at the bytecode of the following MVP.

class Outer {
    private Outer() {
    }

    public static class Inner {
        new Outer();
    }
}

Let's compile.

javac Outer.java

When compiling an inner class, Java will actually create a separate class file for the inner class. The name of this class is <Name of outer class>$<Name of inner class>. We can see this in the output

# ls | grep class
Outer.class
Outer$1.class
Outer$Inner.class

There's the expected outer class, the inner class. There is also an extra classfile here called Outer$1.class.

As a note, if you change the constructor to protected and compile again you'll notice something's missing. The Outer$1.class file only exists when the private constructor is accessed. This is a clue as to how javac resolves the fact that we need to access a private member.

So, that $1 class is getting generated all because of Java. Let's take a peek at it. To do this, we use the tool javap that comes with each JDK installation.

javap -p -s -v <class>

• -p asks the tool to show private, protected, and public methods.
• -s includes the internal type signatures
• -v asks it to be verbose. This is good when you want to really dig in. You'll get the constant pool & the bytecode of the methods.

# javap -p -s -v Outer$1
class Outer$1
  minor version: 0
  major version: 52
  flags: ACC_SUPER, ACC_SYNTHETIC
Constant pool:
<<<snipped>>>
{
}
SourceFile: "Outer.java"
EnclosingMethod: #6.#0                  // Outer
InnerClasses:
     static #1; //class Outer$1

See those opening, and closing braces? That's javap's idea of the body of the class. In this case, there are no methods defined at all.

Do note the ACC_SYNTHETIC, that means this class does not exist in the source code. Keep that in mind for later. Any time you see this flag, it means the Java compiler created something for us that we didn't directly write.

So this class seems to do absolutely nothing, yet is always created when accessing a private member from an inner class. This indicates that this is at least quite important to this functionality.

To find out, let's check out those other classes, starting with the Outer class.

# java -p -s -v Outer
public class t.Outer
  minor version: 0
  major version: 52
  flags: ACC_PUBLIC, ACC_SUPER
Constant pool:
<snipped>
{
  private t.Outer();                          // First constructor
    descriptor: ()V
    flags: ACC_PRIVATE
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: invokespecial #2                  // Method java/lang/Object."<init>":()V
         4: return
      LineNumberTable:
        line 4: 0
        line 6: 4

  t.Outer(t.Outer$1);                         // Second Constructor
    descriptor: (LOuter$1;)V
    flags: ACC_SYNTHETIC
    Code:
      stack=1, locals=2, args_size=2
         0: aload_0
         1: invokespecial #1                  // Method "<init>":()V
         4: return
      LineNumberTable:
        line 3: 0
}

And there we go, there's ACC_SYNTHETIC on the second constructor. Let's break down interesting things about this:

• The flags do not include privacy. That means we're package private.
• The flags include ACC_SYNTHETIC so we definitely didn't write this constructor.
• The descriptor attribute references the $1 class in the following way:
  • L means an Object parameter, what comes next is the class. Outer$1;
  • V is a void return. This is expected.

The body of the method is relatively simple as it;

• Loads the self reference
• Calls the private constructor
• Finally returns

What this shows, is that javac generated us a constructor that allows for classes in the same package to call the private constructor via this synthetic constructor. The descriptor (method params + return) change is clever as it ensures that there isn't a collision between the regular functions and the ones that javac generated. We know this for sure because Outer$1 is a synthetic class and thus can't have been referenced in any prior source.

One more class to go. Let's check out the call site in the inner class.

# javap -p -s -v Outer\$Inner
public class t.Outer$Inner
  minor version: 0
  major version: 52
  flags: ACC_PUBLIC, ACC_SUPER
Constant pool:
<snipped>
{
  public Outer$Inner();
    descriptor: ()V
    flags: ACC_PUBLIC
    Code:
      stack=3, locals=1, args_size=1
         0: aload_0
         1: invokespecial #1                  // Method java/lang/Object."<init>":()V
         4: new           #2                  // class Outer
         7: dup
         8: aconst_null
         9: invokespecial #3                  // Method Outer."<init>":(LOuter$1;)V
        12: pop
        13: return
}

Alright, so let's beak down the constructor.

• 0: Load this
• 1: Invoke the constructor for the superclass Object
• 4: Create a new instance of the Outer class
• 7: Duplicate the new instance on the stack. This is required for the invokespecial step.
• 8: Load null into the stack
• 9: Call the synthetically created constructor
• 12: Pop the stack
• 13: Return

If you're wondering why there are breaks between the numbers, the labels at the start are byte offsets since the start of the method. An instruction can take more than 1 byte (eg, new takes 2). So the jumps in the numbers are due to the fact that the instruction prior has taken more than a single byte.

So, when you have a

• public
• protected
• package private

method, javac has do to bugger all. It's when it's private that we have to jump through these hoops.

Who can call that?

So what gives Inner the right to call that synthetic method? Is there a way that you can write code that'll allow a non-inner class to call it?

On the surface it seems the only requirements are to be able to call the constructor that takes the Outer$1 as a parameter and to be in the same package.

A difficulty arises though, as javac will not allow you to compile a java source file that references the $1 class. Or at least, I've been unable to.

There's nothing from stopping us from creating our own bytecode though. Perhaps we can create a class that'll allow us to open up this private constructor to the world.

To do this, we can use the java-asm library. This library is a bytecode manipulation framework. You can use it to manipulate existing class files, or even just create your own classes. Let's take our knowledge of how the inner class calls the synthetic method and utilise that.

Looping back for a moment, the core part of the bytecode of the synthetic call was

1: new Inner
2: dup
3: aconst_null
4: invokedynamic Inner.<init>(LOuter$1;)V
5: pop
6: return

What we want to do is insert that bytecode into a method of another class. In java-asm, that looks as follows. Here we're creating a class called Example.

        ClassWriter cw = new ClassWriter(0);
        cw.visit(V1_1, ACC_PUBLIC, "Example", null, "java/lang/Object", null);

        MethodVisitor mw = cw.visitMethod(ACC_PUBLIC, "<init>", "()V", null,
                null);
        mw.visitVarInsn(ALOAD, 0);
        mw.visitMethodInsn(INVOKESPECIAL, "java/lang/Object", "<init>", "()V", false);
        mw.visitInsn(RETURN);
        mw.visitMaxs(1, 1);
        mw.visitEnd();

Firstly, we must create our constructor. Javac will create a constructor when compiling a source file, however when writing bytecode we must create a constructor explicitly.

Now, let's create another method that does contain the bytecode in the same way the inner class has.

        mw = cw.visitMethod(ACC_PUBLIC + ACC_STATIC, "createBankAccount",
                "(I)V", null, null);

        mw.visitTypeInsn(NEW, "BankAccount");
        mw.visitInsn(DUP);
        mw.visitVarInsn(ILOAD, 0);
        mw.visitInsn(ACONST_NULL);
        mw.visitMethodInsn(INVOKESPECIAL, "BankAccount", "<init>", "(LBankAccount$1;)V", false);
        mw.visitInsn(RETURN);
        mw.visitMaxs(3, 2);
        mw.visitEnd();

        byte[] code = cw.toByteArray();

        FileOutputStream fos = new FileOutputStream("Example.class");
        fos.write(code);
        fos.close();

We then write out the class by running the above snippets together.

We should be able to verify this class is valid by using javap. If for some reason the class file was generated unsuccessfully, javap will complain that there's some issues. Here's the output, not including the method bodies

# javap -v -s -p Example
...
public static void createBankAccount(int);
  descriptor: (I)V
  flags: ACC_PUBLIC, ACC_STATIC
  Code:
    stack=3, locals=2, args_size=1
       0: new              #13        // class BankAccount
       3: dup
       4: iload_0
       5: aconst_null
       6: invokespecial    #15        // Method BankAccount."<init>":(ILBankAccount$1;)V
       9: return
...

We should now be able to use our class file.

class Runner {
    public static void main(String... args) {
        Example.createBankAccount(100_000);
    }
}

We then run javac Runner .java. This should compile successfully! Doing so shows we've been able to create code that has compiled against our generated bytecode.

From here, run it with java Runner.

# java Runner 
Your bank balance is $100000

Your console should now print the amount that you've asked for, even though it the constructor is private. To the jvm, you've only called a package-private method, and that's considered perfectly fine.

The implications of this is that if you're able to insert your own class into a package containing a outer/inner pair, then you're also able to call any private parts of the outer class that the inner class references.

This also holds for private methods. In my instance, calling a private method will create an access$xxx synthetic method, where x are digits. These are created somewhat deterministically, so a rogue user would have little difficulty figuring out what to call when generating their own class file.

Finally, there's no requirement for any part of the inner class to be public for this technique to be utilised. An entirely private inner class still opens up the holes in the exact same way. This can lead to a false sense of security if steps aren't taken to mitigate this approach.

Overall

When utilising the outer/inner class pattern, you have to be prepared for the fact that javac is creating holes in the privacy of certain parts of your code. This is by design, when querying Oracle they said it is a known and expected behaviour. So this isn't a crazy security problem in the JDK, it's just an interesting side effect of how Java manages to create a pattern that we all very much enjoy when using patterns such as the Builder.

If you're running an application, this could only be a problem if you run user-code, and allow classes to be defined in pre-existing packages.

To mitigate this, when running untrusted code, you should ensure that the code does not define itself in a pre-existing package. Of course, if you're running untrusted code, this is probably number 100,000 on the priority listing of possible actual issues and this is mostly an exercise into the possibilities.

For now though, celebrate the fact that you've managed to call a private method/constructor (in another one of the many ways). That's pretty cool, and hopefully you've learnt a little bit about how Java abstracts these language features.

• 1.5 cups flour - high gluten
• teaspoon salt
• pinch of sugar
• 5/8 cup water (warm) + 1 tsp yeast

knead, mix, etc. Frozen, then thawed for later.

Pizza

• half red onion (raw)
• tomato paste
• 1 chicken breast (cooked)
• 1/2 capsicum (small cuts)
• Chilli Olive Oil
• 80g fetta
• Mozzarella
• Bush tomato sauce (chilli)

Method

1. Ensure the BBQ is on High, try have it at 200^oC
2. Put down a flour surface, put dough on top
3. Punch dough, pushing out onto the bench
4. Flip the dough a couple of times, pushing out even more
5. Once dough is around half the size of the final pizza, pick up
6. Start with stretching the dough, try to make sure there exists a nice round crust. Use knuckles to stretch.

Now for the pizza toppings

1. Put on tomato sauce first, followed by a small amount of mozzarella. Less than normal
2. This is followed in order by:

• Capsicum
• Red onion
• Chicken

1. Crumb the fetta over the pizza using your fingers to make small chunks
2. Drizzle olive oil over the top, add tomato sauce as desired
3. Cook on pizza stone in the BBQ until crust is firm on top.

Why should you never store secrets or passwords in a String in Java?

This can be answered based on a few facts of Java. First of all, java.lang.String is immutable. Therefore, once allocated, there is no way to change the value of a String. So this means if you ever update the value, simply an entire new String is created. The old string merely sits in memory with no JVM reference to it.

Lets start a look at this problem by investigating the class DumpMemory (no points for what we'll do later). It stores a password that's meant to be entered and matched.

class DumpMemory {
    private String line = "hi there";

    public void setPassword(String value) {
        managerPassword = value;
    }

    public boolean matches(String password) {
        return managerPassword.equals(password);
    }
}

Enter our bad actor. Somehow, they got access to the machine that it runs on. Lets just pretend that's not impressive already and instead highlight another security concern.

Our bad actor can now core dump out our running Java process

# gcore <pid> 

From here it's trivial to get the Strings referenced by each class.

First, lets create a hprof file. This will transform the dump file created earlier into a filetype that Java specific tools are happy to read.

> jmap -dump:format=b,file=dump.hprof $JAVA_HOME/bin/java <core file>

We can take a look at this using the jhat tool.

> jhat dump.hprof
Reading from dump.hprof...
Dump file created Wed Mar 29 11:32:54 AEDT 2017
Snapshot read, resolving...
Resolving 10814 objects...
Chasing references, expect 2 dots..
Eliminating duplicate references..
Snapshot resolved.
Started HTTP server on port 7000
Server is ready.

If we navigate to http://localhost:7000 we can see the package we're interested in.

After selecting DumpMemory we then select the a list of references to our object.

then we can select the DumpMemory reference.

Suddenly, plain as day, we can see our String just sitting there. This is entirely expected and understandable. The String exists as a field on an instance of a class, and thus has to stay around while the instance is in scope. Given the JRE isn't going to encrypt Strings at rest, we have to expect it to be in our heap dump.

Though couldn't we just lower the scope of the String as much as possible and wait for GC to come along and clean up our secret?

Lets do so in the following example:

public class DumpMemory {

    String line = "";

    public void run() {
        line = new Scanner(System.in).nextLine();

        System.out.println("You entered: " + line);
    }

    public static void doSomething() {
        new DumpMemory().run();
    }

    public static void main(String[] args) {
        doSomething();
        new Scanner(System.in).next();
    }
}

Again, a somewhat contrived example. We get a String from the user. Lets pretend this is a secret String. We proceed to securely print it, and then doSomething() completes. This means that the instance that was created in that method is now up for garbage collection, as it is no longer in scope.

In the example, once doSomething() has completed we wait for user input so we can get a chance to dump the process.

Once gcore has completed, we can't use the standard Java core analysers. That's because they'll ignore parts of the dump that aren't part of a valid stack/heap. Instead, we can take a look at the core file in a hex viewer.

When running this, I entered the String beagle3 as the input. Now analysing the core dump, you might expect that since the object has gone out of scope you wouldn't see it any more.

Hmm. Even though the String is firmly out of scope at the point that we took the dump, look what we actually see. The reasoning behind this is the fact that going out of scope is not enough for the JVM to get rid of the underlying data.

What about after a GC?

Sometimes you'll read an answer about the fact that if you use a String rather than a byte[] then you'll have to wait for garbage collection to come along before that String will no longer exist in memory.

Lets investigate that claim to find if that's always true.

This time we'll start Java with a slightly custom flag. java -verbose:gc dva.DumpMemory. This will allow us to see when gc has taken place so that we can know when to take our dump.

Here's the output of a run of the same program

# java -verbose:gc dva.DumpMemory                                                                
beagle3
You entered: beagle3
[GC (JvmtiEnv ForceGarbageCollection)  13107K->1544K(251392K), 0.0033103 secs]
[Full GC (JvmtiEnv ForceGarbageCollection)  1544K->1399K(143360K), 0.0099899 secs]

What I did was first enter the hyper-secure password, and then using JProfiler, I forced a full GC of the program. You can see the effects in the JProfiler memory view.

Taking the dump once again. I opened it in a hex editor.

This dump was performed after the GC, but the String is still sitting in memory. The reason this is happening is due to the fact that, when garbage collecting, the JVM does not bother to zero out memory (for good reason). Instead it simply discards GC'able objects from its internal memory model and marks the memory as available for future object storage.

Will it ever go away?

Yes!

Lets make an example that does just that. In the previous example we had a single GC, lets change it so we're having lots of GC.

public class DumpMemory {
    String line = "";

    public void run() {
        line = new Scanner(System.in).nextLine();

        System.out.println("You entered: " + line);
    }

    public static void doSomething() {
        new DumpMemory().run();
    }

    public static void main(String[] args) {
        doSomething();
        
        while(true) {
            new Date().toString();
        }
    }
}

As we run this one we're going to see GC occurring consistently without any JProfiler help

# java -verbose:gc dva.DumpMemory                                                                
beagle3
You entered: beagle3
[GC (Allocation Failure)  65536K->840K(251392K), 0.0010497 secs]
[GC (Allocation Failure)  66376K->848K(251392K), 0.0010727 secs]
[GC (Allocation Failure)  66384K->896K(251392K), 0.0006899 secs]
[GC (Allocation Failure)  66432K->872K(316928K), 0.0008372 secs]
[GC (Allocation Failure)  131944K->880K(316928K), 0.0010596 secs]
[GC (Allocation Failure)  131952K->856K(438272K), 0.0009543 secs]
[GC (Allocation Failure)  263000K->840K(438272K), 0.0014307 secs]
[GC (Allocation Failure)  262984K->816K(427520K), 0.0004112 secs]
[GC (Allocation Failure)  252720K->816K(418816K), 0.0003833 secs]
[GC (Allocation Failure)  242992K->816K(408576K), 0.0003985 secs]
[GC (Allocation Failure)  233776K->816K(400896K), 0.0004971 secs]
[GC (Allocation Failure)  225072K->816K(391680K), 0.0004864 secs]
[GC (Allocation Failure)  216880K->816K(384512K), 0.0004790 secs]
[GC (Allocation Failure)  208688K->816K(375808K), 0.0004398 secs]
[GC (Allocation Failure)  201008K->816K(369664K), 0.0004333 secs]
[GC (Allocation Failure)  193840K->816K(361472K), 0.0003493 secs]
[GC (Allocation Failure)  186672K->816K(355840K), 0.0004877 secs]

Now if we take a dump here, we're going to (almost definitely) be unable to find the string beagle3. This is because as the String containing beagle3 went out of scope, the memory it existed in was available to be used at some point in the future. As we continually allocate objects, eventually the place where beagle3 sat in memory is overwritten with a new object that was allocated when creating a Date.

This means that eventually our mistake of having a secret as a String is occluded, but if your application only performs GC every now and then, you are liable for having that secret around for much longer than you wanted in your lexical scope.

What should I do instead?

Use byte[]. Byte arrays are the underlying way that Strings are stored in Java anyway. You should be able to perform the same kind of changes on a byte array that you would otherwise perform on a String. You'd simply change

public boolean checkSecret(String otherText) {
    // assuming 'secret' is a String
    return secret.equals(otherText);
}

to

public boolean checkSecret(byte[] otherText) {
    return Arrays.equal(secret, otherText);
}

The great thing that a byte array gives you is the chance to zero out your array as soon as you've finished using the secret. This means you have explicit controls over the lifetime of your secret.

// Secret starts existing in memory
byte[] superSecret = passwordManager.loadPassword();

boolean eq = Arrays.equal(superSecret, userInput);

// Secret is purged from memory
Arrays.fill(superSecret, (byte) 0);
return eq

Unlike altering a String, when you alter a byte array you are actively changing the underlying data in the heap. Therefore you don't run into the same issue where your data will stay long after the secret has gone out of scope. Even if the memory representing superSecret does stay around for a while, it is only filled with zeros in our example and thus has hidden our secret from prying eyes (unless your secret was actually all zeros).

So, if you ever are dealing with Strings that should be very private you should follow these rules

• use byte[] in every place that you wanted to use a String
• Only load the byte[] data right before you need it
• As soon as the critical work is done, zero out your byte array

Using this will give you the smallest window of time that your secret is sitting in memory, waiting for someone to discover it.

Lets prove this

Taking the following program

package dva;

import java.security.SecureRandom;
import java.util.Arrays;
import java.util.Random;
import java.util.Scanner;

public class DumpByteMemory {

    // This is an unrealistic random, but it allows us to
    // look for what could be considered a random value
    Random rand = new Random(1);

    byte[] line = new byte[10];

    public void run() throws Exception {
        System.out.println("We're about to generate our *secure key*");
        rand.nextBytes(line);

        System.out.println("Secret is stored, look for it");

        new Scanner(System.in).next();
        Arrays.fill(line, (byte)0);
    }

    public static void doSomething() throws Exception {
        DumpByteMemory f = new DumpByteMemory();
        f.run();
    }

    public static void main(String[] args) throws Exception {
        doSomething();
        System.out.println("Out of scope");
        new Scanner(System.in).next();
    }
}

Here we're pretending a password is being generated, though we're setting a seed so that we know what to look for in the dump. When setting the seed to 1, the example (on my JRE) will produce 73 D5 1A BB D8 9C B8 19 6F 0E for the 10 byte array.

So lets take two dumps:

• One after the message Secret is stored, look for it appears
• One once the message Out of scope appears

Here we see the results of that. At first we see this stream of data in our dump. Then at the second point we see that same memory has been completely zeroed out.

This shows the security that you can get by simply changing the way that you handle your keys. Instead of hoping that GC comes along and eventually overwrites your object you can make sure that you keep your secret around for as short a time as possible.

public class Bird {}

public class Duck extends Bird {}

In this case, Bird is a superclass of Duck. While Duck is a subclass of Bird. In UML this is designated by an arrow pointing from Duck to Bird.

Covariant

Covariant is when the ordering of types is preserved. It orders types from more specific to more generic.

Enumerable<Cat> is a subtype of Enumerable<Animal>. The sub-typing is preserved Enumerable is covariant on <T>

So you're covariant on something. In this case we're covariant on <T>.

An example where your covariant is an array in Java. This is because String[] is a subtype of Object[].

What you can do

Covariance means that you're able to get out the object from the covariant structure. Lets take this in an array.

void extract(List[] arr) {
    List list = arr[0];
    Object ob = arr[0];
}

So due to arrays being covariant you're able to extract values. This is because anything passed in must be a subtype of List[] arr. Therefore we at least have a List[] arr. We may have been passed in anything that is a subtype of List as well.

Therefore we can get out a List and any super-type of List with guarantees that it will not fail.

What you cannot do

Covariance has its problems. Lets take a peek

public static void main(String[] args) {
    String[] names = {"Paul", "Kirren", "Bianca"};
    Object[] casted = (Object[]) names;

	// This is NOT a syntax error
    casted[0] = 1;
}

In this case we've successfully down casted to an Object[]. This is valid because Object[] is the superclass of String[].

The problem occurs when you attempt to insert into the array. Any object at all is allowed into an Object[] array. If the base array was of type Object[] that would be fine! Unfortunately in our case it's of type String[] and thus fails with:

Exception in thread "main" java.lang.ArrayStoreException: java.lang.Integer

Contravariant

Contravariant if the order is reversed from Covariant.

Action<Animal> is a subtype of Action<Cat>

The sub-typing is reversed because Action<T> is contra-variant on T.

import java.util.ArrayList;
import java.util.List;

public class Test {
    public static void main(String[] args) {
        take(new ArrayList<Object>());
        take(new ArrayList<Integer>());

        // This is a SYNTAX error, as it's not a superclass of Integer
        take(new ArrayList<Extension>());
        // So is this, it's totally unrelated to Integer
        take(new ArrayList<String>());
    }

    public static void take(List<? super Integer> list) {
        // I know that the BASE list is of type Integer, or any Super class
        // So what do we get out of this?
        // We are guaranteed to be able to Insert integers!
        list.add(new Integer(5));

         // Note that this does not allow us to insert anything in the super chain
         list.add(new Object()); // This is invalid. We could have a List<Integer>
    }

    private static class Extension extends Integer {
        public Extension(int num) {
            super(num);
        }
    }
}

What we can do!

Remember the previous example, we weren't able to insert because an Object[] may hide the fact the base array is Integer[]. In this case though, take will only accept Lists that will allow Integer to be inserted. You can easily prove this, as the only superclass of Integer is Object. Thus, this can be provided a List<Object> or a List<Integer>.

What we can't do

Now our problem lies in the opposite. With covariance you're always assured that you're able to extract the type in the variable. Contravariance does not provide us with that same guarantee, in fact, getting a variable from a contravariant structure provides no guarantees and can fail.

In this case, imagine take was provided with a List<Object>, you would be unable to take an Integer from it.

So contravariance allows for insertion, but not extraction. This because you know that the container provided knows how to store at least one type in the hierarchy, therefore, your object.

In Summary

Do you need to get able to get objects out? Make sure that your container is covariant.
Do you need to be able be able to insert objects? Make sure it's contravariant.

One was suddenly able to create a secure connection with a person that they had never met before. Previously keys had to be exchanged in some already secure manner such as an in-person exchange. Often this was done in person by exchanging books of keys for use in cyphers. This has obvious downsides, the largest of which being a captured key list will cause all others using the same key list to have their communications suddenly become completely compromised.

This lack of ability to exchange keys upon starting communication meant that in situations such as WWII submarines would often have large amounts of keys stored. That way on their long trips without docking they were able to keep up continuous conversation. A well known example of this is U-559, a German submarine that was forced to surface after the Allies uncovered her location. Upon doing so the Allies instantly attempted to recover the code books. They were successful the code breakers at Bletchley Park were able to break the Naval enigma for several weeks.

Though this is an example which one can definitely make the point that it's a good thing that we didn't have D-H key exchange it does point out the obvious problems that cryptanalisys has been having since rather recently.

The general idea

An often used analogy of the key exchange is colours.

• User A selects a secret colour
• User B selects a secret colour
• User A and B agree on a shared colour
• Users A and B mix their colour and the shared colour
• Users A and B swap their mixed colour
• Users A and B add their own colour to the colour that the other person gave them

After this they will both end up with the same colour. All we have to do is find a mathematical basis that will work the same.

The Security

The security of the above exchange hinges on anything that was made public. In our example it was the two mixed colours that were made public. If an eavesdropper was able to reverse the mixing process they would suddenly be able to figure out each persons' original colour. In our example one can imagine it's rather difficult to remove a colour from a mix. So we just need to make sure that in the mathematics that is used the mixing step will also create a result that makes it difficult to reverse.

The Algorithm

We'll start off with the two traditional characters Alice and Bob. They want to decide on a key that will be used for their encryption.

Diffie-Hellman is not involved in the actual encryption of data. It is only a key exchange algorithm and as such this algorithm only needs to resolve to a shared key that Alice and Bob both can work out.

To start off with they will decide on the a public prime number. So lets say they decided on 29 (p = 29). As well as that a Primitive Root must be decided on. This depends on the prime number, and in our case will be 18 (g = 8).

Both Alice and Bob then will choose their own secret number. This does not have to be prime.

• Alice chooses 8
• Bob chooses 5

They then perform the equation Share = ga mod p where a is their secret number.

• Alice - 88 mod(29) = 20
• Bob - 85 mod(29) = 27

They now share that number with each other. So Alice sends the number 20 to Bob and Bob replies with 27.

They will then perform the following on the number received.

S = ra mod(p)

where r is the number they received.

Alice - 278 mod(29) = 24
Bob - 205 mod(29) = 24

Notice that they both arrived at the result of 24? That will now be the agreed value for their further encryption that takes place. Of course this is just an example and small values were used, in actual implementations they will instead use very large primes and integers.

Security and Validity

Generally maths people will only be keen on using something if they have a proper proof that it works. So lets start by ensuring that we didn't just happen to be lucky when creating our example above. To do this we need to start by defining general numbers.

p = prime decided
g = primitive root

A = Alice's secret number
B = Bob's secret number

Ai = Alice's intemediatry result
   = gA mod(p)
   
Bi = Bob's intemediatry result
   = gB mod(p)

AR = BiA mod(p)
BR = AiB mod(p)

note this is in a working state. Published for editing

Lets get down to business

Lets have a look at an example of RSA before we get into how it works.

In each of these examples we have the following 'actors'.

• Alice - Good guy, friends with Bob
• Bob - Good guy, friends with Alice
• Eve - An eavesdropper. Pretty keen to hear what Alice and Bob are up to.

So to send a message between Alice and Bob we're first going to have to generate our set of public-private keys. The steps for that are below. We'll go through it in more detail in a moment.

• Alice chooses some prime numbers
  
  p = 17
  
  q = 19
• Alice Computes n n = p * q
  
  n = 17 * 19 = 323
  
  Alice can share n with anyone. It's not secret.
• Compute φ(n) = (p-1)(q-1) = 16 * 18 = 288
• Choose an number e such that 1 < e < φ(n)
• gcd(e, φ(n) = 1 therefore e and φ(n) are coprime
• Our example e = 11
  
  Alice can share e with anyone. It's not secret.
• Determine d = e-1 mod φ(n)
• d = 11-1 mod(288)
• Our example makes d = 131

With these numbers we can now make our set of public/private keys.

Public Key

A public key is made up of n and e. n being the multiplication of the two large prime numbers and e being a number between 1 and 288 that had a greatest common divisor with 288 as 1. Often you're fine to just choose a random prime, but do test that gcd(e, φ(n)) = 1 is true.

Alice's public key is (323, 11)

Private Key

Alice's private key is first of all made up with the same n that her public key was made from. The difference is that the other number used for the key is d. This number was the multiplicative inverse of e (modulo φ(n)). We'll go into why this works a bit later but for now you can just solve the equation d = e-1 mod(288). This is done through the Extended Euclid's Algorithm (see below).

Alice's private key is (323, 131)

How to use them

This is where Bob comes in. Bob wants to send Alice the message: you should not trust eve. First Bob knows that any message that he sends must be of an integer value less than n. In this case any message must be less than 228. This counts as 11100100 in binary. So therefore we can set an easy upper bound on only transmitting 7 bits at a time. So lets make our string!

>>> ' '.join(format(ord(x), 'b') for x in "you should not trust eve")
'1111001 1101111 1110101 100000 1110011 1101000 1101111 1110101 1101100 1100100 100000 1101110 1101111 1110100 100000 1110100 1110010 1110101 1110011 1110100 100000 1100101 1110110 1100101'

Encoding using the Public Key

Lets take our first message to send 1111001 and convert it to decimal. Once a decimal we will be able to encode it using the following equation.

Encrypt as follows: CypherText of Message M = Me log(n)

So our binary data can be converted to decimal and will come out as the number 121. We then need to encode this data so that only Alice will be able to read it. Once we do this Bob will not be able to decrypt it again. It's a one way step.

In our example we end up with

c(M) = Me mod(n)
     = 12111 mod(323)
     = 379,749,833,583,241 mod(323)
     = 144

Our first letter is now encoded as 144 or binary 10010000. This can then be sent across the wire to Alice.

Lets Decrypt - Using the Private Key

Alice and only Alice will be able to decrypt the data (assuming that good values were used for the primes originally). So to do that she'll need to perform the following

Decrypt as Plain Text from Message C = Cd mod(n)

m(C) = Cd mod(n)
     = 144131 mod(323)
     = 121

We have just managed to encrypt what is the first letter of our message. We can set this as binary again and convert it back again.

>>> str(unichr(int('01111001', 2)))
'y'

Thus we've managed to send our first letter of our string to Alice. The rest can of course be completed in much the same way.

What makes this strong?

It's all well and good to show that we can go encrypt and decrypt a number. But to prove that it's a good idea we've got to make sure that the public key does not leak any required information. There's a few things that we need to make sure that we can ensure.

• That Bob was definitely using Alice's public key when encrypting
• That Alice was definitely receiving a message from Bob
• That Eve was unable to infer the private key from listening to all public communication to Bob.

Lets start with the last one.

Stopping the Cracker

In our above case there wasn't much that was transmitted publicly. The only information that is available is the public key, and anyone at all can get this. The idea behind a public key is to not keep it safe, it should be able to stand by itself.

With that in mind lets take a look at the information provided in the public key.

• n - The multiplication of the two original primes
• e - A number less than n that is relatively prime number to n

So to get the private key Eve will need to get the factors of n and the number d where d was the multiplicative inverse of e mod n.

N - Why it's hard

So within N are two pieces of information that would unravel the whole thing. If you look at the original process the only numbers that are needed to work out the private key are p, q (the primes used in the original n equation) and e. Seeing we already have e we had better hope that finding out p and q is difficult.

Thankfully. It turns out that it is. It's really, really difficult.

When creating your p and q values each of them is most likely a prime number with a bit length of ~1024. So our number n is going to be incredibly large. This is of prime security concern as we need to make it as difficult as possible to factorise n. If n is ever factorised then suddenly we've lost all of our security as the private key is trivial to figure out.

The problem of Integer Factorisation is a difficult problem. Without the use of Quantum computer (and Shor's algorithm) we are unable to currently solve this in a respectable time. Most of the methods that do work are based around trying a heap of values. For this reason we are able to be fairly sure that if we choose strong primes in p and q that the key will not be cracked (at least for a few thousand millennia).

Interactive Example!

n = x =

φ(n) = (-1)(-1) =

Now we need to choose 1 < e < φ(n) and gcd(e, φ(n)) = 1;
We'll choose a common e that's used. That being 65,537 which is 2¹⁶+1

Values

N:

φ(n):

E:65537

D:

Public Key

(e,n) = (, )

Private Key

(d,n) = (, )

Git hooks are often run as a bash script. For the hobby programmer or this may be a bit annoying to have to learn how to use Bash just to make sure each commit will always include a meaningful commit message linked with a JIRA message, or to take a restart a server after files have been updated.

Thankfully it's practical and simple to make the git hook gracefully hand off the grunt work to a python script (or any other language for that matter).

Getting Started

Today we're going to delve into the client side git hooks. They come in several flavours all meant for hooking into different parts of the process.

• pre-commit Runs before you've even entered your commit message. Here you can take a look at the current snapshot. You're able to check for all sorts of code requirements here such as code style, tests passing, builds running, etc. A non-zero return from a script would abort the commit.
• prepare-commit-msg This will run before the user is asked for their commit message. It allows a hook to edit the default message to provide to the user. This would enable you to create a commit that would set the default message as something like the classes the classes that have been added. A non-zero return from a script would abort the commit. This hook is provided the following parameters:
• Reference to file that holds the commit message so far. This can be edited to be changed to whatever you're after.
• Type of commit (merge, fast forward, etc)
• SHA-1 of the commit
• commit-msg Runs prior to a commit being completely verified but after the commit message has been created. This will run with a single parameter which is a file reference to the commit message. A non-zer exit value will abort the commit.
• post-commit Runs once the commit has been completely finalized. A non-zero return from this hook will not cause the commit to fail. This is mainly used for notification purposes for that reason.

Example - commit-msg Hook

Unsurprisingly we have a small sidebar for the Windows users to get this working. As windows doesn't recognise the shebang (#!) at the start of a file we will only have the option of writing the git hook in a way that cmd.exe will understand. So to get started we create this small file in your base directory of your poject.

.git/hooks/commit-msg

#!/bin/sh
python .git/hooks/commit-message.py

This file can still be used on Linux/Mac (as to why we've put the shebang in the start still). This will then mean that before any commit is created and placed in the git log the pre-commit-hook.py file will run. The return code for the python script will then be used as the overall return code for the script.

In all further references to .git/hooks/commit-msg you can assume we're talking about .git/hooks/commit-msg.py if you're using Windows.

Using Python

So lets open up our python file and create a small tool that will perform some basic action for us.

.git/hooks/commit-msg

#!/usr/bin/env python

import sys

print "Starting commit-msg hook"

sys.exit(0)

This example will exit with a system code of 0. Zero indicates success, any non zero value will indicate that the commit is unsuccesful. When creating a commit now we will get the following output:

paul@laptop ~/gitProjects/easygitwithpython $ git add *
paul@laptop ~/gitProjects/easygitwithpython $ git commit -m "Hi there"
Starting pre-commit hook
[master b5254f4] Starting commit-msg hook
 1 file changed, 1 insertion(+), 1 deletion(-)
paul@laptop ~/gitProjects/easygitwithpython $ 

Getting more useful

We can extend our previous example to make use of the commit data. An example of this is making sure that a commit message is above a certain length and matches a regular expression for a JIRA issue.

.git/hooks/commit-msg

import sys, re

#Required parts 
requiredRegex = "[A-Z]{2,}-\\d+"
requiredLength = 15

#Get the commit file
commitMessageFile = open(sys.argv[1]) #The first argument is the file
commitMessage = commitMessageFile.read().strip()

if len(commitMessage) < requiredLength:
    print "Commit message is less than the required 15 characters."
    sys.exit(1)
    
if re.search(requiredRegex, commitMessage) is not None:
    print "A JIRA issue must be linked with a commit"
    sys.exit(1)

print "Commit message is validated"
sys.exit(0)

The prepare-commit-msg git hook will have a single argument, that is a file reference to the commit message that has been set. This hook runs before the commit has fully been validated and as such a non-zero return code will still cause the git commit to fail.

You can see the following confirmation of this hook

paul@laptop ~/gitProjects/easygitwithpython $ git commit -m "short"
Commit message is less than the required 15 characters.
paul@laptop ~/gitProjects/easygitwithpython $ git commit -m "A long message, no JIRA issue"
A JIRA issue must be linked with a commit
paul@laptop ~/gitProjects/easygitwithpython $ git commit -m "A long message, with JIRA-123"
Commit message is validated
[master 0ecdd0d] A long message, with JIRA-123
 1 file changed, 1 insertion(+)
paul@laptop ~/gitProjects/easygitwithpython $ 

Another windows caveat. When using commit hooks that require passing through the argument to the Python file you'll need to have your prepare-commit-msg to pass through those arguments manually.

.git/hooks/commit-msg

#!/bin/sh
python commit-msg.py $1

This will pass through the first argument to Python and will get you to the same level that Unix systems are up to.

Using Server Side Hooks

One problem that comes with hooks is that you are not able to enforce them on any client who has cloned your repository. This may seem annoying but it definitely for good reason. If one was able to enforge git hooks on a repository then you'd be effectively forcing your users to run arbitrary code on their machines.

For this reason it's often better to rely on push hooks. This will allow you to only run the code on a trusted place. So with that in mind, we can delve into our possible server side git hooks. They are as follows.

• pre-receive - Gets a list of references that are being changed. A non-zero return will cause all to be rejected.
• update - Runs for each branch that is being pushed. A non-zero return will cause the branch individually to be rejected
• post-receive - Runs once the commits have been succesfully pushed. Usually used for notifications or for deploying to a production site.

We'll go further into server side hooks in a further post

I love Android.

I also think that testing is super duper.

For this reason I've always been a big fan of attempting to find ways to perform better testing. Whether this be through use of continuous integration, mocking or even sweet as dependency injection coupled with Gradle.

Simple Gradle Example

Gradle allows very easily to allow for different flavours or build configurations. A very small example that shows a small change in builds is below. It is an exerpt from a larger Android build.gradle that will set various build configurations based on the build type.

buildTypes {
    release {
        signingConfig signingConfigs.release
        minifyEnabled false
    }
    debug {
        minifyEnabled false
        debuggable true
}

A quick look at Gradle will make it fairly obvious that we're using a whole lot less boilerplate than the standard Maven POM. We are managing to leverage the Groovy DSL to be more expressive in fewer characters. Surprisingly XML isn't the best thing for all things configuration based.

The above buildTypes will set Android debugging flags. Based on the type of release that is being done, it will switch on debugging and sign the app correctly.

Adding some Android flair

Gradle is a powerful language that means that you're able to very easily perform some simple actions that will affect the global state of your program. The following code example shows off a ways to turn on and off the high logging levels.

release {
    buildConfigField "boolean", "HIGH_LOGGING", "false"
}
debug {
    buildConfigField "boolean", "HIGH_LOGGING", "true"
}

This build config field is created in the form

buildConfigField <type>, <name>, <value>

We will then be able to use this at any point as a static field in our program via BuildConfig.VARIABLE_NAME. This makes it suddenly incredibly easy to create a logging stub that will allow for easy debug logging. Note that this part of the Android Gradle plugin and does not work out of the box for a Java build.

public class Logger{

    public static void log( Level logLevel, String message ){
        if( logLevel >= Level.WARN ){
            log.log( logLevel, message );
        }
        else if( BuildConfig.HIGH_LOGGING )
            log.log( logLevel, message );
        }
    }
}

Suddenly we will have a call to a logger that will always be able to choose how much to log based on the build type. This means whenever you release your apk you will never have a problem of creating a bucket of log files that are over the top but there is ZERO extra configuration that is required when developers are attempting to debug on their own devices.

But we don't have to stop there. Gradle will also let us perform more advanced actions such as those that will include or exclude whole source packages automatically. This allows for you to perform fantastically simple mocking of objects that would otherwise be a pain to switch in and out.

Controlling source packages with Gradle

Android isn't always easy to perform testing on. This is generally for the same reason that UI testing in the web isn't exactly simple either. For this people generally use products such as HTMLUNIT or other selenium test runners. If you're after something similar Robolectric allows you to perform Unit testing with the ability to test as though a UI existed.

Sometimes though that's not enough. When testing apps it's common that the only way that you can find exactly what you need to improve by continually attempting to use your app. This is known as dogfooding and is used extensively in software companies. We can leverage Gradle to allow us to create different APK files for the internal development team. We can combine build types and build flavours to allow us to create builds specifically for tablets, for smaller phones, for paid versions, for free versions. This without too much hair falling out. So lets demonstrate how it would be possible to create app versions that contained debug/release only code.

Getting build specific control

Gradle is run through Groovy. This means that we can technically do anything that we can do in Groovy. Suddenly this means our build system can have logic rather than just a lot of configuration steps. Instead of a barrage of tags all over the place we can instead create dynamic builds, even including user input. This would be fantastic if you wanted to tag a build with a special name. We'd be able to do something like:

android {
    defaultConfig {
        applicationVariants.all { variant ->
            def file = variant.outputFile
            variant.outputFile = new File(file.parent, "supercoolfilename.apk")        
        }
    }
}

Which you can extend this to get the user input to do the following.

def console = System.console()

def newname = console.readLine("Set the APK name >")

android {
    defaultConfig {
        applicationVariants.all { variant ->
            def file = variant.outputFile
            variant.outputFile = new File(file.parent, newname + ".apk")
        }
    }
}

We will now get a console output that will ask us first to confirm the apk name before it actually will get built. But this is really not something that we'll want unless we're releasing (maybe like what Google does with android for mako and other build types). So you can easily extend this to make it totally based on the build version. (Reference)

def console = System.console()
android {
    applicationVariants.all { variant ->
        if (variant.buildType.name.equals("debug")) {
            def apk = variant.outputFile;
            variant.outputFile = new File(apk.parentFile, "debugAndroidApp.apk");
        } else if(variant.buildType.name.equals("release")) {
            def apk = variant.outputFile;
            variant.outputFile = new File(apk.parentFile, "AndroidApp.apk");
        }
    }
}

Suddenly we have a way to have build names for each of our releases all in the same number of lines that the standard Maven system would have managed to add two dependencies.

Using Gradle To Deploy Alternative Files

So up to this point we've done some fairly small things with the Gradle system. It will actually allow us to override class implementations so that we may go ahead and inject different requirements based on our test system. In the following example we're going to use it to inject a sidebar into a fake android app that will allow us to change what we inject into our app. Through this we're able to add things such as a Http Service that fails requests 50% of the time.

To start off with, we'll have a look at the directory structure to create this example:

src  -\
     - main
     - release
     - debug

All code within the main folder will always be compiled for use. This is code that will exist in both our release and debug builds. The release and debug folders will each be compiled when the relevant gradle build is taking place. The generic way that this is used is android.sourceSets.<flavour><ReleaseType>, so in our case we do not have a flavour so we just use the release type. Therefore android.sourceSets.release and android.sourceSets.debug are the two that we are using. This corresponds to src/release and src/debug. Every instance of these extends the standard android.sourceSets.main which will be in src/main.

To leverage this we will create, a MainActivityExtender which will be implemented in both release types. This will be used to add extra functionality to the MainActivity in the main source set. A comparison of the files will show the small difference that lies between them.

MainActivity.class

class MainActivity extends Activity {

    public SideBarManager(Context ctx, View parentView){
        // Inflate the new View
        setContentView(R.layout.MainActivity);
        
        runDebugPostHook();
    }
    
    public void runPostHook(){
        // Nothing exists here in the MainActivity version
    }
}

/src/debug/net/devlsash/MainActivityExtender.class

class SidebarManagerExtender extends MainActivity {
    
    @Override
    public void runPostHook() {
        // Now we should create new details based on the 
        // fact that we're running in Debug mode
        
    }
    
}

/src/release/net/devlsash/MainActivityExtender.class

class SidebarManagerExtender extends MainActivity {
    // We will not override runPostHook here
}

Android Manifest

<activity
    android:name="net.devslash.testapp.RunningMainActivity"
    android:label="@string/app_name"
    android:windowSoftInputMode="adjustResize|stateHidden">
    <intent-filter>
        <action android:name="android.intent.action.MAIN" />
        <category android:name="android.intent.category.LAUNCHER" />
    </intent-filter>
</activity>

This may all seem a bit over the top. Couldn't we just do as we did before? Set a flag and perform an if statment to do the same thing? That's completely correct. This is merely what I'd call cleaner and less prone to errors or possible leaking of debug features to the final application.

One thing not demonstrated here is the ability to override xml files. This provides extra utility due to the fact that instead of the following

Button someButton; 
if(Flag.DEBUG){
    setContentView(R.layout.debugMainActivityLayout);
    someButton = (Button) findViewById(R.id.debugButton);
} else {
    setContentView(R.id.MainActivityLayout);
}
// Then possibly later on
if(someButton != null) {
    // Do something here
}

With the Gradle source replacement/swapping in we're able to simplify it to

setContentView(R.layout.MainActivityLayout);

Then you can place different xml files in the directory src/main/res/layout/MainActivity.xml and src/debug/res/layout/MainActivity.xml. Based on the gradle build type we will override the MainActivity.xml with our debug version whenever it is built.

Then leveraging what we did before in regards to having an extended MainActivity we would be able to access the debug only parts of the XML in our post hook.

Overall Gradle gives a large amount of build configuration steps that otherwise would be a large pain or just take a really long time to do. Gradle's adoption for Android has progressed Gradle a long way as development has continued to move very quickly for a long time. For this reason it's entirely possible that what I've written will be out of date shortly. If you notice this please do leave a comment below and I'll fix up the inaccuracies.

Never trust shared hosting solutions. They generally suck and mean when you have a dead installation of something (maybe even wordpress?) that you're unable to properly recover.

This now sits in the annals of the Microserver where hopefully it will have a little bit more luck.

Moving to Ghost

After reccommendation I've moved from Wordpress to Ghost. Ghost is run on the back of node.js. It doesn't attempt to offer the same things that Wordpress does. But for that reason it's perfect for what i'm attempting to acheive here. Ghost wants to be a blogging site. Wordpress wants to be a "Web Operating System"

when you think about it, we’re kind of building a web operating system.

-Matt Mullenweg

This can be easily compared with the ideology of Ghost. It's plainly pronounced on their main entry to the site.

Well that's what I want.

And thus continues my experiment into the foray of technical blogging. I'm currently creating a writeup of the use of gradle in regards to testing with a focus on Android. I should have it finished in a couple of days.