What are Python exception reference cycles

A Python exception has a reference cycle when it has a reference chain back to itself (i.e. “exc -> … -> exc”). The following code shows an example of the cycle:

1
2
3
4
5
6
7
8
9

def inner():
    exc = ValueError("")
    raise exc

try:
  inner()
except Exception as exc:
  assert exc.__traceback__.tb_next.tb_frame.f_locals['exc'] is exc
  print("Found Python exception reference cycle!")

Why are Python exception reference cycles bad

To understand why Python exception reference cycles are bad, we need to understand some behaviors of Python GC and Python exceptions.

Python GC

Python has two ways to find objects to deallocate: reference counting finds objects whose ref counts are zero and the garbage collector finds objects with circular references. For exceptions with reference cycles, only the garbage collector can deallocate them.

Unless manually triggered by gc.collect(), Python GC runs every X (allocations - deallocations). Unlike other languages, it’s not triggered under memory pressure so it’s purely count based. The implication is that a program can hold lots of memory in a few large objects with reference cycles and doesn’t trigger GC because the count threshold isn’t reached. In the worst case, the program can OOM.

Python exceptions

Unlike many other languages, a raised Python exception has references to all frames of the call stack from main all the way to where the exception is raised, and each frame has references to all local variables. As a result, a Python exception can keep lots of objects alive (i.e. prevent ref counts from going down to zero).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

def inner():
  big = [1] * 1024 * 1024
  exc = ValueError("")
  raise exc

def outer():
  large = [2] * 1024 * 1024
  try:
    inner()
  except Exception as exc:
    assert exc.__traceback__.tb_next.tb_frame.f_locals['big'] == [1] * 1024 * 1024
    assert exc.__traceback__.tb_frame.f_locals['large'] == [2] * 1024 * 1024
    assert exc.__traceback__.tb_frame.f_back.f_locals['huge'] == [3] * 1024 * 1024
    print("Exception has reference to big, large and huge objects!")

huge = [3] * 1024 * 1024
outer()

The net effect is that an exception with a reference cycle can hold lots of objects alive until the next GC happens which can be greatly delayed and the program may just OOM or run out of resources (e.g. open files) before that.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

import gc

class Giant:
  def __init__(self):
    self.data = [4] * 1024 * 1024

  def __del__(self):
    print("Giant __del__ is called")

def inner():
  exc = ValueError("")
  raise exc

def outer():
  giant = Giant()
  try:
    inner()
  except Exception:
    pass

outer()
print("Calling GC")
gc.collect()

If you run the above program, you will get

1
2

Calling GC
Giant __del__ is called

and this shows that we now rely on GC to deallocate the Giant object which is undesirable.

How to avoid Python exception reference cycles

It turns out the issue we described above is well known to the Python developers and there is a PEP that’s implemented to help us break the exception reference cycle. Basically we should avoid having local variables point to the exception. In our example, we can do the following to break the reference cycle:

1
2
3
4
5
6
7
8
9
10
11
12
13

def inner(): # BAD, has cycle
  exc = ValueError("")
  raise exc

def inner(): # GOOD, no cycle
  raise ValueError("")

def inner(): # GOOD, no cycle
  exc = ValueError("")
  try:
    raise exc
  finally:
    del exc

What is smaps

The smaps file in Linux provides detailed information for each of the process’s VMAs (Virtual Memory Areas), represented by struct vm_area_struct. The content of smaps file of a process is constructed by iterating through each VMA and calling show_smap. The Shared_Clean, Shared_Dirty, Private_Clean and Private_Dirty fields are printed here and calculated here.

Shared vs Private

From the perspective of smaps, whether a page is considered shared or private is determined by its page_mapcount which is basically page->_mapcount NOT by the MAP_SHARED or MAP_PRIVATE flags used in mmap. _mapcount of a page is a counter tracks how many page table entries (PTEs) point to this physical page across ALL processes. A page is considered shared if there are at least 2 PTEs point to it.

Let’s verify this with different cases:

1

dd if=/dev/zero of=/tmp/smaps  bs=1M count=1024

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>

int main(int argc, char *argv[]) {
  int fd = open("/tmp/smaps", O_RDWR);
  if (fd < 0) {
    perror("Failed to open");
    exit(1);
  }

  const int num_mmaps = atoi(argv[1]);
  const bool map_shared = strcmp(argv[2], "MAP_SHARED") == 0;
  const bool map_populate = strcmp(argv[3], "MAP_POPULATE") == 0;
  printf("Command line args: num_mmaps=%d, map_shared=%d, map_populate=%d\n",
         num_mmaps, map_shared, map_populate);

  for (int i = 0; i < num_mmaps; i++) {
    void *map = mmap(NULL, 512 * 1024 * 1024, PROT_READ | PROT_WRITE,
                     (map_shared ? MAP_SHARED : MAP_PRIVATE) | (map_populate ? MAP_POPULATE : 0),
                     fd, 0);

    if (map == MAP_FAILED) {
      perror("Failed to mmap");
      close(fd);
      exit(1);
    }

    int *data = (int*) map;
    printf("data = %d\n", data[0]);
  }

  while (1) {
    sleep(2000000);
  }
}

This is a test program that mmaps 512M out of a 1GB file in various ways.

Single MAP_SHARED mmap

1

./test 1 MAP_SHARED NO_MAP_POPULATE

Content of smaps:

1
2
3
4
5
6
7
8
9
10
11

7f418625d000-7f41a625d000 rw-s 00000000 103:01 132                       /tmp/smaps
Size:             524288 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Rss:                  64 kB
Pss:                  64 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:        64 kB
Private_Dirty:         0 kB
...

Several things to note here:

• The mmaped pages are considered private even though we use MAP_SHARED because each physical page only has 1 PTE points to it.
• mmap is lazy and doesn’t populate the page table unless MAP_POPULATE is specified and that’s why we didn’t see 524288 KB in Private_Clean.
• We also didn’t see 4 KB in Private_Clean even though we just read one integer from the file and that’s due to fault-around.

Multiple MAP_SHARED mmaps in single process

1

./test 2 MAP_SHARED NO_MAP_POPULATE

Content of smaps:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

7f764be9a000-7f766be9a000 rw-s 00000000 103:01 132                       /tmp/smaps
Size:             524288 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Rss:                  64 kB
Pss:                  32 kB
Shared_Clean:         64 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:         0 kB
...
7f766be9a000-7f768be9a000 rw-s 00000000 103:01 132                       /tmp/smaps
Size:             524288 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Rss:                  64 kB
Pss:                  32 kB
Shared_Clean:         64 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:         0 kB
...

In this case, we mmaps the same file twice in the same process so each page now has 2 PTEs point to it and that’s why they are considered shared.

Multiple MAP_SHARED mmaps in multiple processes

1

./test 1 MAP_SHARED NO_MAP_POPULATE

1

./test 1 MAP_SHARED NO_MAP_POPULATE

Content of smaps:

1
2
3
4
5
6
7
8
9
10

7fb76d49e000-7fb78d49e000 rw-s 00000000 103:01 132                       /tmp/smaps
Size:             524288 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Rss:                  64 kB
Pss:                  32 kB
Shared_Clean:         64 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:         0 kB

1
2
3
4
5
6
7
8
9
10

7f4e1eb61000-7f4e3eb61000 rw-s 00000000 103:01 132                       /tmp/smaps
Size:             524288 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Rss:                  64 kB
Pss:                  32 kB
Shared_Clean:         64 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:         0 kB

In this case, we mmaps the same file once in two processes so each page also has 2 PTEs point to it and that’s why they are considered shared.

Multiple MAP_PRIVATE mmaps in single process

1

./test 2 MAP_PRIVATE NO_MAP_POPULATE

Content of smaps:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

7f726c909000-7f728c909000 rw-p 00000000 103:01 132                       /tmp/smaps
Size:             524288 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Rss:                  64 kB
Pss:                  32 kB
Shared_Clean:         64 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:         0 kB
...
7f728c909000-7f72ac909000 rw-p 00000000 103:01 132                       /tmp/smaps
Size:             524288 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Rss:                  64 kB
Pss:                  32 kB
Shared_Clean:         64 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:         0 kB
...

Since MAP_PRIVATE is copy-on-write (COW) and we only read from the file so the two VMAs are backed by the same physical pages and hence those pages are considered shared.

Multiple MAP_PRIVATE mmaps with MAP_POPULATE in single process

1

./test 2 MAP_PRIVATE MAP_POPULATE

Content of smaps:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

7fafbaabe000-7fafdaabe000 rw-p 00000000 103:01 132                       /tmp/smaps
Size:             524288 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Rss:              524288 kB
Pss:              524288 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:    524288 kB
...
7fafdaabe000-7faffaabe000 rw-p 00000000 103:01 132                       /tmp/smaps
Size:             524288 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Rss:              524288 kB
Pss:              524288 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:    524288 kB
...

If the mmap is PROT_WRITE, MAP_PRIVATE and MAP_POPULATE, then kernel will not only populate the page tables but also eagerly triggers COW so each VMA is backed by their own private physcial pages.

Clean vs Dirty

From the perspective of smaps, whether a page is considered clean or dirty is determined by this. Conceptually a page is dirty if it cannot just be discarded without data loss, which means it has dirty writes that haven’t been flushed to the underlying file or swap space yet.

Neural Networks

LLMs are large neural networks so having a basic understanding of what neural networks are is helpful.

Victor Zhou’s Neural Networks From Scratch

Neural Networks From Scratch is a 4-post series that introduces classic neural networks, recurrent neural networks (RNNs) and convolutional neural networks (CNNs). It doesn’t require any prior knowledge except for some math. One good thing about this series is that it’s very hands-on and you will learn, step-by-step, how to write a simple nueral network from scratch using only numpy to solve a real problem.

Michael Nielsen’s Visual proof that neural nets can compute any function

Visual proof that neural nets can compute any function gives a visual explanation of the universal approximation theorem that neural networks can be used to approximate any continuous function to any desired precision.

LLM

Andrej Karpathy’s Neural Networks: Zero to Hero

Neural Networks: Zero to Hero is a hands-on course that starts with building a simple neural network from scratch and ends with building a GPT from scratch. It’s a great course to learn LLM progressively.

Inference

The best way to learn LLM inference is writing an inference engine from scratch IMO and this helps you better understand model architectures as well.

Andrej Karpathy’s llama2.c

llama2.c is an inference engine for Llama 2 in one file of pure C. It can give you a taste of what LLM inference looks like. Based on it, I wrote a Rust implementation with optimizations like tensor parallelism.

Managerial Leverage

Managerial output = Output of organization = A1 * L1 + A2 * L2 + … where A is a managerial activity and L is the corresponding leverage. Manager activities include information-gathering, information-giving, decision-making, nudging, and being a role model. Leverage, by definition, is the measure of the output generated by any given managerial activity. Based on the formula, a manager can increase his output by:

1. Increase the rate with which a manager performs his activities, speeding up his work.
2. Increasing the leverage associated with the various managerial activities.
3. Shifting the mix of a manager’s activities from those with lower to those with higher leverage.

The art of management lies in the capacity to select from the many activities of seemingly comparable significance the one or two or three that provide leverage well beyond the others and concentrate on them. This is how we should allocate our time —— the single most important resource that we allocate from one day to the next. Given time is a finite resource, and when we say “yes” to one thing we are inevitably saying “no” to another, we should be very selective about things we choose to do.

Meetings —— The Medium of Managerial Work

There are two kinds of meetings. The first is process-oriented meeting, knowledge is shared and information is exchanged. Such meetings take place on a regularly scheduled basis. The second is mission-oriented meeting, the purpose is to solve a specific problem and produce a decision. They are ad hoc affairs, not scheduled long in advance, because they usually can’t be.

One-on-one is a process-oriented meeting. It should be regarded as the subordinate’s meeting, with its agenda and tone set by him.

For a mission-oriented meeting, the purpose should be decided beforehand and it’s the meeting chairman’s job to make sure the meeting accomplishs the purpose for which it was called. Since the goal of this type of meetings is to make a decision, we should control the number of attendees. A decision-making meeting is hard to keep moving if more than six or seven people attend. Eight people should be the absolute cutoff. Decision-making is not a spectator sport, because onlookers get in the way of what needs to be done. Once the meeting is over, the chairman must nail down exactly what happened by sending out minutes that summarize the discussion that occurred, the decision made, and the actions to be taken. And it’s very important that attendees get the minutes quickly, before they forget what happened. The minutes should also be as clear and as specific as possible, telling the reader what is to be done, who is to do it, and when.

Decisions, Decisions

An organization does not live by its members agreeing with one another at all times about everything. It lives instead by people committing to support the decisions and the moves of the business.

The ideal decision-making process involves three stages. The first stage should be free discussion, in which all points of view and all aspects of an issue are openly welcomed and debated. The greater the disagreement and controversy, the more important becomes the word free. The next stage is reaching a clear decision. Again, the greater the disagreement about the issue, the more important becomes the word clear. Finally, everyone involved must give the decision reached by the group full support.

The person makes the decision is the DRI and everyone else needs to disagree and commit once the decision is made. The DRI should have the confidence to make the decision especially when consensus is not reached and the self-confidence mostly comes from a gut-level realization that nobody has ever died from making a wrong business decision, or taking inappropriate action, or being overruled (most decisions are two-way doors).

We shouldn’t enter the decision-making stage too early or wait too long. If we make the decision too early, we may miss some opinions. If we make the decision too late (aiming for the absolutely right decision), we are not moving fast enough. Often, it’s better to try out a 60-point decision and iterate on it quickly based on feedbacks than trying to find a 100-point decision endlessly.

For decision-making, one of the manager’s key tasks is to settle six important questions in advance:

• What decision needs to be made?
• When does it have to be made?
• Who will decide?
• Who will need to be consulted prior to making the decision?
• Who will ratify or veto the decision?
• Who will need to be informed of the decision?

Planning: Today’s Actions for Tomorrow’s Output

Planning is to answer the qeustion: What do I have to do today to solve —— or better, avoid —— tomorrow’s problem? Today’s gap represents a failure of planning sometime in the past.

MBO (management-by-objectives) is a planning process that can be applied to daily work. A successful MBO system needs only to answer two questions:

• Where do I want to go? (The answer provides the objective).
• How will I pace myself to see if I am getting there? (The answer gives us milestones, or key results.)

The Sports Analogy

The role of the manager is that of the coach. First, an ideal coach takes no personal credit for the success of his team, and because of that his players trust him. Second, he is tough on his team. By being critical, he tries to get the best performance his team members can provide. Third, a good coach was likely a good player himself at one time. And having played the game well, he also understands it well.

Performance Appraisal: Manager as Judge and Jury

Performance reviews is the single most important form of task-relevant feedback we as supervisors can provide. The goal is to improve the subordinate’s performance instead of cleansing our system of all the truths we may have observed about the subordinate so less may very well be more (give one feedback and one area for improvement at a time).

It might be counterintuitive but we should spend more time trying to improve the performance of our stars. After all, these people account for a disproportionately large share of the work in any organization. Put another way, concentrating on the stars is a high-leverage activity: if they get better, the impact on group output is very great indeed. We must keep in mind, however, that no matter how stellar a person’s performance level is, there is always room for improvement.

Install pyrasite

1

pip install pyrasite

pyrasite allows you to attach to a running Python process and run arbitrary Python code. It needs ptrace to function properly and the way to enable it varies depending on your OS. For Ubuntu, you can do:

1

sudo sysctl -w kernel.yama.ptrace_scope=0

If you use Conda, you might need to run unset LD_LIBRARY_PATH so that gdb can use the system libstdc++.so instead of the one installed inside your conda env which can be incompatible.

Install guppy3

1

pip install guppy3

guppy3 has a subpackage heapy that allows you to inspect the heap.

Dump Heap

Once everything is installed, we can then use pyrasite to attach to the target running Python process:

1

pyrasite-shell <pid>

This attaches to the process and opens a REPL that you can run the heap dump code using guppy3:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

from guppy import hpy
h = hpy()
heap = h.heap()

print(heap.all) # print the heap dump
"""
Partition of a set of 222038 objects. Total size = 26540189 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0  67095  30  6591256  25   6591256  25 str
     1  50143  23  3623400  14  10214656  38 tuple
     2  14415   6  2563320  10  12777976  48 types.CodeType
     3  28124  13  2205321   8  14983297  56 bytes
     4   2175   1  2064360   8  17047657  64 type
     5   4816   2  1962952   7  19010609  72 dict (no owner)
     6  13577   6  1846472   7  20857081  79 function
     7   2175   1  1087024   4  21944105  83 dict of type
     8    690   0  1019024   4  22963129  87 dict of module
     9  10472   5   312536   1  23275665  88 int
    10    273   0   244608   1  23520273  89 google._upb._message.MessageMeta
"""

print(heap.all[10].shpaths) # print the shortest path from root to object with index 10
"""
 0: hp.Root.i0_modules['ray._private.gcs_utils'].__dict__['ActorTableData']
 1: hp.Root.i0_modules['ray._private.gcs_utils'].__dict__['AvailableResources']
 2: hp.Root.i0_modules['ray._private.gcs_utils'].__dict__['ErrorTableData']
 3: hp.Root.i0_modules['ray._private.gcs_utils'].__dict__['GcsEntry']
 4: hp.Root.i0_modules['ray._private.gcs_utils'].__dict__['GcsNodeInfo']
 5: hp.Root.i0_modules['ray._private.gcs_utils'].__dict__['JobConfig']
 6: hp.Root.i0_modules['ray._private.gcs_utils'].__dict__['JobTableData']
 7: hp.Root.i0_modules['ray._private.gcs_utils'].__dict__['PlacementGroupTableData']
 8: hp.Root.i0_modules['ray._private.gcs_utils'].__dict__['PubSubMessage']
 9: hp.Root.i0_modules['ray._private.gcs_utils'].__dict__['ResourceDemand']
<... 271 more paths ...>
"""

There are lots of urgent things (whirlwind) to do every day to keep the lights on. Failure to do so can kill the business today. However, there are also important things to take the business to the next level. Failure to do those can kill the business tomorrow. As a result, being able to execute important things successfully while doing the day job is critical for the success of the business in the long term. To help teams to do so, this book presents a methodology called 4DX that consists of four disciplines.

Discipline 1: Focus on the Wildly Important

The key of the first discipline is focus. Attempting to spread the limited capacity across multiple goals is the most common cause of failure in execution. Instead, one and only one wildly important goal (WIG) should be identified and executed.

To identify the WIG, one can ask the following question: If every other aspect of our team’s performance remained at its current level, what is the one area where significant improvement would have the greatest impact? The resulting WIG should be specific, clearly measurable and in the form of From X to Y by when (e.g. improve the NPS score from 1 to 6 by end of the year).

Discipline 2: Act on the Lead Measures

The key of the second discipline is leverage. Lag measures are the tracking measurements of the WIG and they tell you whether you achieve the WIG or not. For example, if the WIG is improving the NPS score from 1 to 6 by end of the year then the lag measure is the NP score. The issue with the lag measure is that you cannot influence it directly and it is lagging in the sense that by the time you get the result, it is already too late and the performance that drove it is in the past. In contrast, lead measures measure things that lead to the WIG and can be influenced by us directly. For example, burning 1000 calories every day is a good lead measure for a WIG of losing 20 pounds in 3 months.

Two basic characteristics of a valid lead measure are predictive and influenceable: it’s predictive of achieving the WIG and can be influenced directly. There could be many lead measures for a WIG and finding the highest-leverage ones is perhaps the toughest and most intriguing challenge for leaders trying to execute a WIG. Finding lead measures requires the involvement of the frontline team members and that involvement brings engagement and commitment. In the end, it’s the frontline team that acts on the lead measures. A leader can only veto but not dictate what the final lead measures should be. The resulting lead measures are the team’s bet: they bet that by driving these lead measures, the team is going to achieve the WIG.

Discipline 3: Keep a Compelling Scoreboard

The key of the third discipline is engagement. A compelling player’s scoreboard that clearly shows whether a team is winning or losing drives the highest level of engagement and the engagement drives the highest level of performance. The fundamental purpose of a player’s scoreboard is to motivate the players to win.

A compelling scoreboard has the following four characteristics:

• It’s simple as a player’s scoreboard in a sporting event, not a more complex coach’s scoreboard.

• It can be seen easily. A visible scoreboard makes sure that the WIG and lead measures are not forgotten in the constant urgency of day-to-day responsibilities.

• It shows both the lead and lag measures. From the scoreboard, the team can see what they are doing (the lead measures) and what they are getting (the lag measures). Once the team sees that the lag measure is moving because of the efforts they have made on the leads, it has a dramatic effect on engagement because they know they are directly impacting the results.

• It tells if a team is winning or losing immediately. If the team can’t quickly determine if they are winning or not by looking at the scoreboard, then it’s not a game, it’s just data.

Discipline 4: Create a Cadence of Accountability

The key of the fourth discipline is accountability. This is where execution actually happens. The previous three disciplines set up the game but until this discipline is applied, the team is not in the game. Without this discipline, there will always be actions the team members know they should perform but never actually do with real consistency due to whirlwind.

To create a cadence of accountability, a WIG session is held on the same day and at the same time every week with the following agenda:

1. Account. Report on last week’s commitments.

2. Review the scoreboard. Learn from successes and failures.

3. Plan. Clear the path and make new commitments.

To figure out the high-impact commitments for the week, ask the question “What are the one or two most important things I can do this week to impact the lead measures?” Once the commitments are made from team members, they are unconditional regardless of the whirlwind and the team member is accountable to each other.

Finally, a successful 4DX not only helps the team to achieve the specific WIG but also produces a winning team that performs at the next level and is ready for the next WIG.

Human-centered Design

A good design should be human-centered. It should put human needs, capabilities, and behavior first, then accommodate those needs, capabilities, and ways of behaving. The design should adapt to people, not the opposite.

A good design should be understandable and usable. The design should help users establish a concept model of the system that is an explanation, usually highly simplified, of how the system works. There are seven design principles to achieve a good design:

• Discoverability: It is possible to determine what actions are possible and the current state of the device.
• Feedback: There is full and continuous information about the results of actions and the current state of the product or service. After an action has been executed, it is easy to determine the new state.
• Conceptual model: The design projects all the information needed to create a good conceptual model of the system, leading to understanding and a feeling of control. The conceptual model enhances both discoverability and evaluation of results.
• Affordances: The proper affordances exist to make the desired actions possible.
• Signifiers: Effective use of signifiers ensures discoverability and that the feedback is well communicated and intelligible.
• Mappings: The relationship between controls and their actions follows the principles of good mapping, enhanced as much as possible through spatial layout and temporal contiguity.
• Constraints: Providing physical, logical, semantic, and cultural constraints guides actions and eases interpretation.

The Double-Diamond Model of Design

Finding the right problem and finding the right solution are the two components of design and this corresponds to the two phases of the design process. The double-diamond model of design describes these two phases.

To find the real, ROOT problem to solve, we can use the five whys approach. We should keep asking why until we find the real, fundamental, root problem. Don’t rely on whatever people tell you because they will tell you that they want “a faster horse”. In fact, the author has a somewhat counterintuitive rule for himself: never solve the problem I am asked to solve. Because the problem being asked to solve, invariably, is not the real, fundamental, root problem.

No Human Error, Only Bad Design

Human error usually is a result of poor design. Whenever human errors happen, we should think about how we can design better to eliminate those errors or reduce the impact if complete elimination is impossible. This is an important mindset to have: if a system lets you make the error, it’s badly designed.

There are several design practices to prevent human errors:

• Avoid procedures that have identical opening steps but then diverge.
• Ensure that controls and displays for different purposes are significantly different from one another.
• Try to avoid modes. If they are necessary, the equipment must make it obvious which mode is invoked.
• People will be interrupted during their activities and they may need assistance in resuming their operations.
• Don’t count on much being retained in people’s short-term memory. The most effective way of helping people remember is to make it unnecessary.

Successful Product, Successful Design

The author argues that the design is successful only if the final product is successful —— if people buy it, use it and enjoy it. It doesn’t matter how great the design is if people don’t buy it.

To create a successful product, people should focus on strengths, not weaknesses. If the product has real strengths, it can afford to just be “good enough” in the other areas. People should also focus on the true needs of the people who use the product and ignore competing voices.

To turn a good idea into a successful product requires timing. Good ideas that appear too early will fail even if eventually others introduce them successfully. It takes a long time (may be decades), for good ideas to traverse the distance from conception to successful products. We should ride the wave to make the product more likely to be successful.

Technology Adoption Life Cycle

For the technology or product offered by a company, its adoption goes through several phases from left to right.

Innovators (Technology Enthusiasts)

This is a group of people who are techies. They just like new technologies for its own sake and are willing to try them. They don’t represent a significant market in themselves, nor do they have enough buying power. However they are still important to win over because 1) they are the sounding board and test bed for the new technology or product 2) they are the reference base for early adopters.

Early Adopters (Visionaries)

This group of people treat the new technology or product as an opportunity to have a strategic leap forward. They want fundamental breakthrough enabled by the new technology and are willing to take high risks due to the potential order-of-magnitude return on investment. These people have big budget to implement their strategic initiative so they are an important source of high-tech development capital.

Early Majority (Pragmatists)

These people represent the mainstream market that any startup wants to win due to its volume. Unlike visionaries, they want incremental, predictable improvements instead of disruptive ones. They have the following characteristics: 1) they are loyal once won and will even help you to defend new comers 2) they want to buy the whole product from the market leader 3) they reference other pragmatists not visionaries.

Late Majority (Conservatives)

They buy the extremely mature product with low cost just to stay on par with the rest of the world. They are the low-margin end of the market but have high volume.

Laggards (Skeptics)

Even though they may not buy products, they do point continually to the discrepancies between the sales claims and the delivered product. Startups should use their feedbacks to continuously improving the products.

Word of Mouth

In the high-tech buying process, word of mouth is the number-one source of information that buyers reference. People reference each other during the buying decision.

Whole Product

In the mainstream market, people want to buy whole product not core product. In other words, they want to buy the product that has a surrounding ecosystem, which radically reduces their burden of support. Since whole products grow up around the market leading products and not around the others, pragmatists buy from market leaders.

Chasm

The chasm between visionaries and pragmatists exists because visionaries is a very poor reference base for pragmatists. They are fundamentally different groups of people for the following reasons: 1) visionaries don’t need many references to buy a product, rather they want to be the first to buy the product and create competitive advantage; pragmatists, on the other hand, need extensive reference to prove the validity of the new product 2) visionaries care more about the future, in fact, they are defining the future, while pragmatists don’t put a lot of stake in futuristic things 3) visionaries don’t expect the existence of the whole product and they are willing to piece together one themselves in return for getting a jump on their competition, while pragmatists only want to buy the whole product 4) visionaries, successful or not, don’t plan to stick around long while pragmatists are cautious about their decisions since they know they will have to live with the results. This effectively means that when we try to win over pragmatists, there is effectively no reference base to start with.

D-Day

The way to cross the chasm successfully is by launching a D-Day type of invasion focusing on a highly specific target segment within a mainstream marketplace.

Target the Point of Attack

The first thing to do is finding the target niche market segment which is also called the beachhead segment. The segment should be big enough to matter, small enough to win and good fit with your crown jewels. Customers in that segment should have compelling reason to buy the product or in other words, feel enough pain. Instead of picking the optimal beachhead to be successful, which is very hard if not impossible, what more important is winning whatever good enough beachhead that’s picked.

It’s very important to pick a very specific beachhead to conquer instead of the entire mainstream market when we are crossing the chasm for the following reasons: 1) word-of-mouth has boundaries which is ususally within a market segment so winning over one or two customers in each of five or ten different segments is worse than winning four or five customers in one segment in terms of word-of-mouth effect 2) it’s easier to become a big fish in a small pond and achieve market leadership given that pragmatists want to buy from market leaders 3) our scarce resource is only enough to build the whole product for a single niche market.

Assemble the Invasion Force

The next thing is creating the ecosystem around our core product, namely the whole product. This often requires bringing in partners and allies needed to make it a reality.

Define the Battle

Then we need to create the competition and positioning in order for our product to be easy to buy. Viable competition is important since where there is no competition, there is no market. Market alternatives indicate the existence of the budget dollars to buy our products and product alternatives call out the differentiation. The positioning of our product should be short enough to pass the elevator test so that it can successfully create and occupy a space inside the target customers’ head.

Launch the Invasion

Finally we need to select the intended distribution channel and set pricing to give us motivational leverage over that channel.

Multipliers are leaders who make people better and more capable. They can access and revitalize the intelligence in the people around them. They are genius makers by accessing and multiplying the genius in others. They have a growth mindset, which is a belief that basic qualities like intelligence and ability can be cultivated through effort. In contrast, diminishers drain intelligence and capability out of the people around them.

Multiplier Mindset

What people believe affects their behaviors. As a result, we need to have a multiplier mindset first in order to become one. The fundamental assumption a multiplier has is that people are smart. They discover the genius in people by asking “how is this person smart?”. Multipliers believe that:

1. People are smart and will figure things out.
2. If I can find someone’s genius, I can put them to work.
3. People’s best thinking must be given, not taken.
4. People get smarter by being challenged.
5. With enough minds, we can figure it out.

Challenger vs Know-it-all

Since multipliers believe that people get smarter by being challenged, they ask really insightful and interesting questions that make people think. They don’t limit the team to what they know, they push their teams beyond their own knowledge and that of the organization. Even if they have the answer, the don’t just give them. Instead, they just provide enough information to provoke thinking and to help people discover and see the opportunity for themselves. What’s more important as a leader is not having the right answer but asking the right questions.

Debate Maker vs Decision Maker

Multipliers believe that with enough minds, we can figure it out. As a result, they like collective debate. Through debate, they challenge and stretch what people know, thus making the organization smarter over time and creating the organizational will to execute the decisions made. In contrast, when people execute an undebated decision, they turn to debating the soundness of a decision rather than executing it.

Liberator vs Tyrant

Multipliers believe that people’s best thinking must be given, not taken. They provide a safe environment for people to think and make mistakes. The highest quality of thinking cannot emerge without learning and learning can’t happen without mistakes. Intimidation and fear rarely produce truly great work. Speaking of creating a safe environment, there is no easier way to invite experimentation and learning than to share stories about multipliers’ own mistakes.

While multiplier gives space and safe environment, they also demand the best work. It’s a fair trade. They create an intense environment that requires people’s best thinking and their best work. They generate pressue, but they don’t generate stress. Requiring people’s best work is different from insisting on desired outcomes. Stress is created when people are expected to produce outcomes that are beyond their control. But they feel positive pressure when they are held to their best work. Multipliers distinguish best work from outcomes.

Investor vs Micro-manager

Multipliers believe that people are smart and will figure things out. So they operate as investors, giving ownership that keeps rolling back to other people. As investors, they define ownership, invest resources and hold people accountable. When they teach, they invest in their people’s ability to solve and avoid problems in the future. In the end, multipliers enable others to operate independently by giving other people ownership for results and investing in their success. They create organizations that can perform and win, not only without them on the field, but long after their direct influence is felt.

Talent Magnet vs Empire Builder

Multipliers believe that if I can find someone’s genius, I can put them to work. They are like magnets that draw in talent and develop it to its fullset. They look for talent everywhere, find people’s native genius, utilize people at their fullsets and remove the blockers. Multipliers not only notice people’s talent, they label it for them. By telling people what they see, they raise people’s awareness and confidence, allowing them to provide their capability more fully. What’s more, once they uncover the native genius of others, they look for opportunities that demand that capability.

All incidents are gifts

Incidents are opportunities for us to fix bugs, improve the stability of the system and improve the process of handling incidents. We should treat them as gifts and learn as much as possible out of them.

All incidents should be novel

This basically means that we should never make the same mistake twice. Once an incident happens, we try as hard as we can to fix it and make sure it will never happen again with the same root cause.

Hardware failure is common at scale

We have, on average, 2 DIMM failures per week so we should design our software in a fault tolerant way.

API is sticky

Once clients start to use the exposed APIs, they become extremely sticky. That means we need to design them carefully since changing them afterwards is costly.

Logging is talking to the user/operator

How programs talk to humans has a huge impact on the rate at which mistakes can be fixed. If programs tell humans exactly what is wrong, that rate can be very fast. If programs are silent or overwhelm humans with too much information, that rate can be extremely slow. When there are too many spurious errors, people get alerts fatigue and they will overlook the real problems.

When we write logs in our code, we need to remember that the audience is not just us but also people that may not be familiar with the entire codebase like SREs. That means the log messages should be crystal clear and actionable. Imagining how frustrating it is when oncalls get paged at 2am and they have no clue what those log messages mean and how to act on them. Here is a guideline of how to write good error messages.

Comment on why

Code comments should say why the code is there instead of what the code does. What the code does should be clear from the code itself. If it is not the case, then we should refactor the code to make it clear instead of adding a comment. A large decaying comment is frequently just an apology for crappy code. Don’t accept the apology. Fix the code. Don’t give up until you try your best. Then, as a last resort, write the comment.

Good enough is not enough

The math is simple: 0.8 * 0.8 * 0.8 * ….. = 0. If every time we just achieve good enough, then eventually it will become zero/failure. We should never settle and always try to do as best as we can.

Conduct code review

Code review is not just about finding bugs. We should also think about how we can rewrite the code in a better way that is easily understandable and unlikely to cause future bugs.

Also reading other people’s good reviews allows us to learn not only from our own mistakes but also the mistakes of others.

Write design document

“Writing is nature’s way of letting you know how sloppy your thinking is” by Leslie Lamport. The very act of writing the design document helps to clarify the design itself. It also helps people to learn or understand the system in the future.

Keep your eyes open

Problems are everywhere, it’s just whether we can find them or not. Whenever we are doing some task, it’s not just about finishing the task itself, it’s also about discovering new problems along the way. Here we have a cycle of doing things -> discovering problems -> doing more things. Through this cycle, we make whatever we are building better and better.

If we treat everything we do as an opportunity of discovering new problems, then we will find tons of them. To make this more concrete, let me give some examples:

• Whenever we touch a piece of code, even just one line, it’s an opportunity to look at the surrounding code and see if we can refactor to make it better.
• Whenever we do some repetitive work, it’s an opportunity to automate it.
• Whenever we talk to clients, it’s an opportunity to learn their pain points.

Amar Bose also told an interesting story where the student with the eyes open saw a huge opportunity while doing a quite tedious work.

Keep your eyes open, problems and opportunities are around you.

Lower Processing Time & Higher Throughput

If we reduce the processing time, the throughput might be higher. For example, the throughput is 10 requests per second if the processing time is 100ms CPU time assuming it’s a single CPU system. If the processing time is reduced to 10ms CPU time, the throughput is increased to 100 requests per second.

Higher Processing Time & Lower Throughput

This is the opposite of lower processing time & higher throughput. This is undesirable since we lose both processing time and throughput.

Lower Processing Time & Lower Throughput

By optimizing the part of the system that’s not the throughput bottleneck, the throughput keeps the same or is even lower. For example, the request processing time is 100ms CPU time and 10ms IO time. Since the request is CPU bound, reducing the IO time will bring down the overall processing time but throughput will still be the same: 10 requests per second. If reducing the IO time comes with the cost of increasing the CPU time (say CPU time becomes 101ms and IO time becomes 5ms), the throughput is actually lower even though the processing time is also lower.

Higher Processing Time & Higher Throughput

If we can reduce the time spent on the bottleneck part of the system at the cost of increasing overall processing time, we can get higher throughput. Lets still use the request with 100ms CPU time and 10ms IO time as an example. If we can reduce the CPU time to 90ms at the cost of increasing the IO time to 30ms, the throughput is higher now even though the overall processing time is also higher.

Higher Waiting Time & Higher Throughput

The maximum throughput we can get is 10 requests per second if the processing time is 100ms CPU time. To achieve that maximum throughput, we need to make sure the CPU is busy all the time (i.e. 100% utilization). In other words, there is always a request in the queue waitting to be processed as soon as the CPU finishes the current request. Basically we want requests to wait for CPU instead of CPU waiting for requests. By having requests wait for CPU, we incur some waiting time to achieve the higher (maximum) throughput.

Lower Waiting Time & Lower Throughput

This is the opposite of higher waiting time & higher throughput. To make sure the waiting time is zero, we need to make sure the CPU is idle and there is no other requests in the queue by the time when the request is received. By having CPU idle for some time, the throughput will be lower than the maximum throughput.

Random Address

Address space layout randomization randomly arranges the address space positions of stack and heap of a process, which means the address of a variable changes for each run. This is the source of nondeterminism if we try to iterate through an unordered container of pointers.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

#include <iostream>
#include <unordered_set>

int main() {
  std::string a = "aaaaa";
  std::string b = "bb";
  std::string c = "cccccccccccc";
  std::unordered_set<std::string*> set;
  set.emplace(&a);
  set.emplace(&b);
  set.emplace(&c);

  for (std::string* p : set) {
    std::cout << *p << "(" << p << ", " << std::hash<std::string*>{}(p) << ")" << std::endl;
  }

  return 0;
}

1
2
3
4
5
6
7
8
9

> clang++ demo.cc -o demo -std=c++11
> ./demo
cccccccccccc(0x7ffeeedbf650, 16564608384425460261)
bb(0x7ffeeedbf678, 5964435063914947271)
aaaaa(0x7ffeeedbf690, 696214236533423747)
> ./demo
bb(0x7ffee3cd6678, 2875564916106665130)
cccccccccccc(0x7ffee3cd6650, 14656991250807877473)
aaaaa(0x7ffee3cd6690, 10500981134548248633)

As we can see, the address of variable a, b and c keeps changing for each run, which affects their positions in the hash set and, as a result, we will iterate them in different orders.

Lets try to disable ASLR and verify that the nondeterminism is gone:

1
2
3
4
5
6
7
8
9

> clang++ demo.cc -o demo -std=c++11 -Wl,-no_pie
> ./demo
bb(0x7ffeefbff678, 6189850235780456993)
cccccccccccc(0x7ffeefbff650, 14250952053958696967)
aaaaa(0x7ffeefbff690, 4593858558987980877)
> ./demo
bb(0x7ffeefbff678, 6189850235780456993)
cccccccccccc(0x7ffeefbff650, 14250952053958696967)
aaaaa(0x7ffeefbff690, 4593858558987980877)

Since the nondeterminism is generally undesired, clang has a checker to detect such usages.

1. Interpreter
2. Transpilation
3. LLVM
4. DynASM

Interpreter

Before showing how to use JIT to run a Brainfuck program, lets see how it can be run by an interpreter.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83

#include <cstdio>
#include <cstdlib>
#include <string>
#include <vector>
#include <fstream>
#include <streambuf>

class BrainfuckInterpreter {
  public:
    BrainfuckInterpreter(const std::string& program)
      : program(program) {}

    void run();

  private:
    const std::string program;
};

void BrainfuckInterpreter::run() {
  char data[30000] = {0};
  char* ptr = data;
  std::vector<size_t> stack;
  size_t skips = 0;

  for (size_t i = 0; i < program.size(); ++i) {
    switch(program[i]) {
      case '>':
        if (!skips) {
          ++ptr;
        }
        break;
      case '<':
        if (!skips) {
          --ptr;
        }
        break;
      case '+':
        if (!skips) {
          ++*ptr;
        }
        break;
      case '-':
        if (!skips) {
          --*ptr;
        }
        break;
      case '.':
        if (!skips) {
          putchar(static_cast<int>(*ptr));
        }
        break;
      case ',':
        if (!skips) {
          *ptr = static_cast<char>(getchar());
        }
        break;
      case '[':
        stack.emplace_back(i);
        if (!*ptr) {
          ++skips;
        }
        break;
      case ']':
        if (*ptr) {
          i = *(stack.end() - 1);
        } else {
          stack.pop_back();
        }
        if (skips) {
          --skips;
        }
        break;
    }
  }
}

int main(int argc, char** argv) {
  std::ifstream ifs(argv[1]);
  std::string program((std::istreambuf_iterator<char>(ifs)), (std::istreambuf_iterator<char>()));
  BrainfuckInterpreter interpreter(program);
  interpreter.run();
  return 0;
}

1
2
3

g++ -std=c++11 -o brainfuck brainfuck.cc

./brainfuck program.bf

For a complicated Brainfuck program like mandelbrot.bf, running in an interpreter will be slow.

Transpilation

One way to generate and run native code for a Brainfuck program is transpilaiton. The first step is generating the equivalent C/C++ code for the Brainfuck program and then using a conventional compiler to generate a shared object. The second step is using dlopen() to load the shared object and run it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96

#include <dlfcn.h>
#include <cstdio>
#include <cstdlib>
#include <string>
#include <vector>
#include <fstream>
#include <iostream>
#include <streambuf>

class BrainfuckTranspiler {
  public:
    BrainfuckTranspiler(const std::string& program)
      : program(program) {}

    void run();

  private:
    const std::string program;
};

void BrainfuckTranspiler::run() {
  std::string ccname = std::tmpnam(nullptr);
  ccname += ".cc";
  std::ofstream ofs;
  ofs.open(ccname);

  ofs << R"(
#include <cstdio>
#include <vector>

extern "C" void _run() {
  char data[30000] = {0};
  char* ptr = data;
  )";

  for (size_t i = 0; i < program.size(); ++i) {
    switch(program[i]) {
      case '>':
        ofs << "++ptr;" << std::endl;
        break;
      case '<':
        ofs << "--ptr;" << std::endl;
        break;
      case '+':
        ofs << "++*ptr;" << std::endl;
        break;
      case '-':
        ofs << "--*ptr;" << std::endl;
        break;
      case '.':
        ofs << "putchar(static_cast<int>(*ptr));" << std::endl;
        break;
      case ',':
        ofs << "*ptr = static_cast<char>(getchar());" << std::endl;
        break;
      case '[':
        ofs << "while (*ptr) {" << std::endl;
        break;
      case ']':
        ofs << "}" << std::endl;
        break;
    }
  }
  ofs << "}";
  ofs.close();

  std::string soname = std::tmpnam(nullptr);
  soname += ".so";
  std::string compile = "g++ " + ccname + " -o " + soname + " -shared -fPIC";
  system(compile.c_str());

  void* handle = dlopen(soname.c_str(), RTLD_NOW);
  if (!handle) {
    std::cerr << dlerror() << std::endl;
  }
  dlerror();
  void (*_run)();
  _run = (void (*)()) dlsym(handle, "_run");
  char* error = dlerror();
  if (error) {
    std::cerr << error << std::endl;
  }
  (*_run)();
  dlclose(handle);

  remove(ccname.c_str());
  remove(soname.c_str());
}

int main(int argc, char** argv) {
  std::ifstream ifs(argv[1]);
  std::string program((std::istreambuf_iterator<char>(ifs)), (std::istreambuf_iterator<char>()));
  BrainfuckTranspiler transpiler(program);
  transpiler.run();
  return 0;
}

1
2
3

g++ -std=c++11 -ldl -o brainfuck brainfuck.cc

./brainfuck program.bf

LLVM

Besides using an external compiler process to generate the native code, we can also use LLVM mcjit by first generating the equivalent LLVM IR for the Brainfuck program. We can generate the IR manually or using clang frontend. Here I’m using clang to generate the IR but LLVM has an example of manually generating it. The LLVM version I’m using is 7.0.1.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129

#include <cstdlib>
#include <string>
#include <vector>
#include <sstream>
#include <fstream>
#include <iostream>
#include <streambuf>
#include <clang/Driver/Compilation.h>
#include <clang/Driver/Driver.h>
#include <clang/CodeGen/CodeGenAction.h>
#include <clang/Frontend/FrontendOptions.h>
#include <clang/Frontend/CompilerInstance.h>
#include <clang/Frontend/CompilerInvocation.h>
#include <clang/Frontend/TextDiagnosticPrinter.h>
#include <llvm/IR/Module.h>
#include <llvm/ADT/StringRef.h>
#include <llvm/IR/LLVMContext.h>
#include <llvm/Support/Host.h>
#include <llvm/Support/TargetSelect.h>
#include <llvm/ExecutionEngine/ExecutionEngine.h>
#include <llvm/ExecutionEngine/GenericValue.h>

class BrainfuckLLVM {
  public:
    BrainfuckLLVM(const std::string& program, const std::string& clangexe)
      : program(program), clangexe(clangexe) {}

    void run();

  private:
    const std::string program;
    const std::string clangexe;
};

void BrainfuckLLVM::run() {
  std::string ccname = std::tmpnam(nullptr);
  ccname += ".cc";
  std::ofstream ofs;
  ofs.open(ccname);

  ofs << R"(
#include <cstdio>
#include <vector>

extern "C" void _run() {
  char data[30000] = {0};
  char* ptr = data;
  )";

  for (size_t i = 0; i < program.size(); ++i) {
    switch(program[i]) {
      case '>':
        ofs << "++ptr;" << std::endl;
        break;
      case '<':
        ofs << "--ptr;" << std::endl;
        break;
      case '+':
        ofs << "++*ptr;" << std::endl;
        break;
      case '-':
        ofs << "--*ptr;" << std::endl;
        break;
      case '.':
        ofs << "putchar(static_cast<int>(*ptr));" << std::endl;
        break;
      case ',':
        ofs << "*ptr = static_cast<char>(getchar());" << std::endl;
        break;
      case '[':
        ofs << "while (*ptr) {" << std::endl;
        break;
      case ']':
        ofs << "}" << std::endl;
        break;
    }
  }
  ofs << "}";
  ofs.close();

  // Generate LLVM IR using clang frontend
  clang::IntrusiveRefCntPtr<clang::DiagnosticOptions> dopts = new clang::DiagnosticOptions();
  clang::TextDiagnosticPrinter* tdp = new clang::TextDiagnosticPrinter(llvm::errs(), &*dopts);
  clang::IntrusiveRefCntPtr<clang::DiagnosticIDs> dids(new clang::DiagnosticIDs());
  clang::DiagnosticsEngine dengine(dids, &*dopts, tdp);

  llvm::Triple triple(llvm::sys::getProcessTriple());
  clang::driver::Driver driver(clangexe, triple.str(), dengine);
  driver.setTitle("clang");
  driver.setCheckInputsExist(false);

  llvm::SmallVector<const char*, 16> args;
  args.push_back("clang");
  args.push_back(ccname.c_str());
  args.push_back("-fsyntax-only");
  std::unique_ptr<clang::driver::Compilation> compilation(driver.BuildCompilation(args));
  const clang::driver::JobList& jobs = compilation->getJobs();
  const clang::driver::Command& command = static_cast<clang::driver::Command>(*jobs.begin());
  const clang::driver::ArgStringList& ccargs = command.getArguments();
  std::unique_ptr<clang::CompilerInvocation> cinvocation(new clang::CompilerInvocation());
  clang::CompilerInvocation::CreateFromArgs(*cinvocation, const_cast<const char**>(ccargs.data()),
      const_cast<const char**>(ccargs.data()) + ccargs.size(), dengine);
  clang::CompilerInstance cinstance;
  cinstance.setInvocation(std::move(cinvocation));
  cinstance.createDiagnostics();

  llvm::LLVMContext context;
  clang::EmitLLVMOnlyAction action(&context);
  cinstance.ExecuteAction(action);
  std::unique_ptr<llvm::Module> module = action.takeModule();

  // Use mcjit to generate the native code from IR and run it
  llvm::InitializeNativeTarget();
  llvm::InitializeNativeTargetAsmPrinter();
  llvm::Function* _run = module->getFunction("_run");
  llvm::ExecutionEngine* ee = llvm::EngineBuilder(std::move(module)).create();
  llvm::GenericValue gv = ee->runFunction(_run, std::vector<llvm::GenericValue>());
  fflush(stdout);

  remove(ccname.c_str());
}

int main(int argc, char** argv) {
  std::ifstream ifs(argv[1]);
  std::string program((std::istreambuf_iterator<char>(ifs)), (std::istreambuf_iterator<char>()));
  BrainfuckLLVM llvm(program, argv[2]);
  llvm.run();
  return 0;
}

1
2
3

g++ -std=c++11 `/path/to/llvm-config --cxxflags --ldflags` -lclangASTMatchers -lclangFrontendTool -lclangFrontend -lclangDriver -lclangSerialization -lclangCodeGen -lclangParse -lclangSema -lclangToolingInclusions -lclangToolingCore  -lclangFormat -lclangIndex -lclangCrossTU -lclangStaticAnalyzerFrontend -lclangStaticAnalyzerCheckers -lclangStaticAnalyzerCore -lclangAnalysis -lclangARCMigrate -lclangRewriteFrontend -lclangRewrite -lclangEdit -lclangAST -lclangLex -lclangBasic `/path/to/llvm-config --libs` -o brainfuck brainfuck.cc

./brainfuck program.bf /path/to/clang

DynASM

Another way of generating the native code is using DynASM from LuaJIT and there is an example online.

1. 移民
2. 全球化
3. 偏见
4. 经济增长
5. 气候变暖
6. 贫富差距

移民

这本书分析了两类移民对当地人的影响，分别是低技能移民（low-skilled immigrants）和高技能移民（high-skilled immigrants）。

很多人认为低技能的移民会伤害到低技能的当地人的利益：工资会降低，工作机会会被抢走。然而研究表明低技能的移民并不会降低当地人的工资和就业率。移民增加了劳动力的供应的同时也增加了劳动力的需求 （One very big problem with the supply-demand analysis applied to immigration is that an influx of migrants increases the demand for labor at the same time it increases the supply of laborers.）。一个原因是低技能的移民会在当地生活消费，从而产生了新的需求，因而创造出更多的工作岗位。比如移民的增加，使得管理者需求增加而本地人因为语言等的优势更容易获得这些岗位。另一个原因是廉价的移民劳动力会延缓机器化自动化的发生。另外低技能移民很多做的都是当地人不想干的活，因此并没有真正从当地人手中抢走任何工作。同时作者认为当地人并不一定会在和移民的竞争中落入下风。相对于一个新来的陌生移民，当地人有着自身的优势，比如语言的优势，老板知根知底的优势。

对于高技能移民，作者认为对当地人的影响是两面的。好的方面是高技能移民能提供廉价的高技能服务比如看病，而这可以使当地的低收入人群受益。不好的方面是当地拥有同等技能的人会面临着更激烈的竞争。

全球化

全球化和自由贸易可以使得每个国家做自己拥有比较优势（comparative advantage）的活，以此提高生产效率（productivity）。这个世界作为一个整体因此受益，而收益的再分配可以让每个人享受到全球化的成果。而为了达到效率的最大化，资源（比如资本和劳动力）需要能够自由流通到需要它们的地方。然而本书却提出了资源往往具有粘性（resources tend to be sticky）的观点，也就是说资源往往到不了最需要它们的地方。比如出于不确定性，人们更愿意留在家乡而不是到更需要劳动力的城市和国家。

同时作者分析了全球化和自由贸易对于发达国家和发展中国家的影响。发达国家的优势是资本而发展中国家的优势是廉价劳动力，因此全球化会使发达国家的资本家受益而发展中国家的工人受益。另一方面，发达国家资本家受益的同时，它们的工人会在和发展中国家廉价劳动力竞争中受损（Workers were likely to suffer from trade in rich countries and benefit from it in poor countries.）。为了帮助受影响的发达国家工人，国家需要进行收入再分配和各类政策扶持。而贸易战却不是解决这个问题的方法之一，它帮助了某些人的同时也会伤害到另外一些人（While trade war may save some jobs in steal, it would likely cause significant new damage to others. The US economy will be fine. Hundreds of thousands of people will not.）。

偏见

本书提到了现在社会充满了对立，分裂和偏见的现象（比如民主党vs共和党，白人vs黑人）。作者认为现在的社交网络非但没有缓解这些现象，反而作为一个回声室（echo chamber），恶化了它们。作者还认为人们的对立和偏见背后有着深层次的原因（比如对世界的不满，感到没有被世界尊重）。打击偏见的方法不应该是直接和他们就此争论或者表现出对他们的鄙视，这样做只会加深偏见（Preferences like prejudice are as much part of the symptoms of the malaise as its cause, perhaps more. Prejudice is often a defensive reaction to the many things we feel are going wrong in the world, our economic travails, and a sense that we are no longer respected or valued.）。为了缓解这些现象，作者认为一个有效的方式是多接触，多互相了解。而学校在这一方面扮演了非常重要的角色。不同背景的年轻人生活学习在一起，可以消除偏见达到互信（Familiarity performed its magic.）。而这也是像哈佛这样培养未来领导人的学校需要招收各个族群学生的一个原因，过多的偏向某一个族群会产生不好的影响（As a flagship producer of the next generation of leaders, Harvard clearly needs to find a place for students from all social groups, and a massive overrepresentation of any particular social group relative to its weight in the population is both perhaps undesirable in a democracy and likely to lead to political problems.）。

经济增长

本书的一个重要观点是人们目前还不知道持久经济增长的内在机制是什么，包括如何恢复增长和保持增长（Despite the best efforts of generations of economists, the deep mechanisms of persistent economic growth remain elusive.）。与此相关的是给富人减税并不会促进经济的增长（Tax cuts for the wealthy do not produce economic growth.）。另一方面，尽管无法带来持续的经济增长，但提高资源利用率，减少资源的错误分配还是能让贫穷国家获得一定的增长，而这需要克服之前提到的资源粘性问题。最后虽然没人知道如何持续不断的增长经济，但所有国家都能致力于改善国民的福祉，即使它们不能被反映在GDP上（GDP values only those things priced and marketed.）。

气候变暖

气候变暖是个全球性的问题，然而它却非常不公平。很大一部分的碳排放都是发达国家导致的（直接排放或者消费发展中国家生产的商品），然而全球变暖的大部分代价却是由发展中国家来承受。因此作者认为发达国家需要为气候变暖承担很大一部分责任。作者还认为人们的消费习惯是可以被改变的，可以变得更低碳更环保。通过税的引导，政府可以慢慢改变人们不环保的消费习惯。

贫富差距

日益扩大的贫富差距也是本书的一个重点讨论议题。作者认为美国这几十年的贫富差距是人为的政策导致的而不是不可控的技术革新（The last thirty years of US history should convince us that the evolution of inequality is not the by-product of technological changes we do not control: it is the result of policy decisions.）。

首先作者认为先富带动后富并没有真正在美国发生，相反富人的财富得到了极大的增长，而穷人的工资涨幅却没有跟上GDP的涨幅，结果就是社会的财富越来越集中到少数富人手中。其中一个重要原因是富人的低税收。低税收使得高工资对CEO们更有吸引力（因为大部分工资都会变成税后工资）而公司之间对优秀CEO的争夺也相应拉高了工资。另外现在CEO主要的收入来自股票而不是基本工资，这也就使得他们对涨工资动力不大，反而会想要降低工资来削减成本从而提升股价（Linking managerial pay to the stock market meant that manager’s pay was no longer linked to a salary scale within the enterprise. When everyone was on the same scale, CEOs had to grow salaries at the bottom to increase their own. With stock options, they had no reason to increase wages at the bottom, and in fact every reason to squeeze costs.）。作者提倡对富人收取高额的税收，并且没有证据表明这样做会降低富人的工作积极性（All in all, therefore, it seems to us that high marginal income tax rates, applied only to very high incomes, are a perfectly sensible way to limit the explosion of top income inequality. They would not be extortionary, since very few people will end up paying them; top mangers will simply not get these kinds of income anymore. And from all we see, they won’t discourage anybody to work as hard as they can.）。另外作者也意识到征收富人税会面临极大的困难。一方面富人有足够的资源对政客进行游说，另一方面富人可以将财富转移到低税收国家进行避税，而国家之间对于这些财富的竞争会使得税率变低（This gives rise to a race to the bottom on tax rates.）。归根到底征收富人税并不是一个经济问题，而是个政治问题（The difficulty of raising top tax rates is a political one.）。

在富人越来越富的同时，美国的穷人却遇到了收入停滞和阶级固化的问题。由此他们会产生绝望或愤怒，绝望的人把问题归咎于自身而愤怒的人把问题归咎于别人（The majority of Americans whose wages and income have stagnated, and who confront an ever-widening gap between the wealth they see around them and the financial woes they are experiencing, face a choice between blaming themselves for not benefitting from the opportunities they believe their society offers and finding someone to blame for stealing their jobs. That way lies despair and anger.）。而这两种情绪对个人和社会都没有益处，绝望的人失去了希望而愤怒的人把问题错误地归咎于移民和全球化（Behind the anti-immigrant views, are two misconceptions: an exaggeration of how many migrants are coming in, or about to come in, and a belief in the nonfact that low-skilled immigrants depress wages.）。作者认为政府需要出台政策来帮助那些受全球化和自由贸易影响的人，其中一个方法就是UBI（universal basic income）。不像别的救助项目，UBI没有任何门槛（比如没有复杂的申请和审核流程）因此可以保证最需要帮助的人能够享受到。另外没有证据显示穷人会因此降低工作积极性或者乱花钱（There is no support in the data for the view that the poor just blow the money on desires rather than needs.）。相反地，这些钱能让穷人少了后顾之忧，让他们有更多的精力放在工作之上，让他们更可能离开家乡去寻找更多的工作机会。在有着这些优势的情况下，实施UBI最大的困难就是政府没有足够的钱。

最后作者充满人文关怀地提到对于穷人的帮助需要照顾他们的尊严，要把他们当作人来对待。他们可能有问题，但他们不是问题本身（While they may have problems, they are not the problem. They are entitled to be seen for who they are and to not be defined by the difficulties besieging them.）。

Murphy’s law

Anything that can go wrong will go wrong.

Amdahl’s law

The performance improvement to be gained by making some part of the execution faster is limited by the fraction of the time that part uses.

Brooks’s law

Adding manpower to a late software project makes it later.

Hofstadter’s law

It always takes longer than you expect, even when you take into account Hofstadter’s Law.

Let’s take float, which has 23 bits mantissa, as an example. 198705381 as an integer has the binary representation 00001011 11011000 00000000 11100101. 198705381.0f as a float has the binary representation 0 10011010 01111011000000000001110. Here a round-down happens due to the default IEEE 754 rounding mode round to the nearest, ties to even and 198705381.0f is rounded down to 198705376.0f.

1

std::cout << std::setprecision(16) << 198705381.0f << "\n"; // output is 198705376

199698905 as an integer has the binary representation 00001011 11100111 00101001 11011001. 199698905.0f as a float has the binary representation 0 10011010 01111100111001010011110. Here a round-up happens and 199698905.0f is rounded up to 199698912.0f.

1

std::cout << std::setprecision(16) << 199698905.0f << "\n"; // output is 199698912

Due to the loss of significance, many surprises can happen:

1
2
3

std::cout << 198705381.0f - 198705380.0f << "\n";  // output is 0 since they are both rounded down to the same value

std::cout << 198705381.0f * 1.005f - 199698905.0f << "\n"; // output is negative (should be positive if we use double instead of float) since one is rounded down and the other is rounded up

References

[1] https://benjaminjurke.com/content/articles/2015/loss-of-significance-in-floating-point-computations
[2] https://www.h-schmidt.net/FloatConverter/IEEE754.html

Setup

1
2

sudo mount -t hugetlbfs -o mode=0777,pagesize=2M nodev /mnt/huge
echo '10' | sudo tee /proc/sys/vm/nr_hugepages

This creates a hugetlbfs with maximum of 10 huge pages.

Example Program

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/mman.h>

#define MB (1024*1024)

void meminfo(char* msg) {
  printf(msg);
  system("cat /proc/meminfo | grep HugePages_");
  printf("\n");
}

int main(int argc, char* argv[]) {
  meminfo("Initial meminfo: \n");

  int fd = open("/mnt/huge/foo.txt", O_RDWR | O_CREAT | O_TRUNC, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH);
  meminfo("After create and open the file: \n");

  ftruncate(fd, 4 * MB);
  meminfo("After set file size to 4MB: \n");

  void* base = mmap(NULL, 4 * MB, PROT_WRITE | PROT_READ, MAP_SHARED, fd, 0);
  meminfo("After mmap 4MB without MAP_POPULATE: \n");

  munmap(base, 4 * MB);
  meminfo("After munmap: \n");

  base = mmap(NULL, 4 * MB, PROT_WRITE | PROT_READ, MAP_SHARED | MAP_POPULATE, fd, 0);
  meminfo("After mmap 4MB with MAP_POPULATE: \n");

  munmap(base, 4 * MB);
  meminfo("After munmap: \n");

  ftruncate(fd, 8 * MB);
  meminfo("After set file size to 8MB: \n");

  base = mmap(NULL, 8 * MB, PROT_WRITE | PROT_READ, MAP_SHARED | MAP_POPULATE, fd, 0);
  meminfo("After mmap 8MB with MAP_POPULATE: \n");

  munmap(base, 8 * MB);
  meminfo("After munmap: \n");

  close(fd);
  meminfo("After close: \n");
}

The output of running this program is:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65

Initial meminfo:
HugePages_Total:      10
HugePages_Free:       10
HugePages_Rsvd:        0
HugePages_Surp:        0

After creat and open the file:
HugePages_Total:      10
HugePages_Free:       10
HugePages_Rsvd:        0
HugePages_Surp:        0

After set file size to 4MB:
HugePages_Total:      10
HugePages_Free:       10
HugePages_Rsvd:        0
HugePages_Surp:        0

After mmap 4MB without MAP_POPULATE:
HugePages_Total:      10
HugePages_Free:       10
HugePages_Rsvd:        2
HugePages_Surp:        0

After munmap:
HugePages_Total:      10
HugePages_Free:       10
HugePages_Rsvd:        2
HugePages_Surp:        0

After mmap 4MB with MAP_POPULATE:
HugePages_Total:      10
HugePages_Free:        8
HugePages_Rsvd:        0
HugePages_Surp:        0

After munmap:
HugePages_Total:      10
HugePages_Free:        8
HugePages_Rsvd:        0
HugePages_Surp:        0

After set file size to 8MB:
HugePages_Total:      10
HugePages_Free:        8
HugePages_Rsvd:        0
HugePages_Surp:        0

After mmap 8MB with MAP_POPULATE:
HugePages_Total:      10
HugePages_Free:        6
HugePages_Rsvd:        0
HugePages_Surp:        0

After munmap:
HugePages_Total:      10
HugePages_Free:        6
HugePages_Rsvd:        0
HugePages_Surp:        0

After close:
HugePages_Total:      10
HugePages_Free:        6
HugePages_Rsvd:        0
HugePages_Surp:        0

Observations

1. Just setting the file size won’t allocate/reserve any huge pages. It only affects the logical file size not the physical one.
2. mmap only reserves the huge pages.
3. Populating the page table or accessing a page actually allocates the huge pages.
4. HugePages_Free includes HugePages_Rsvd so the number of free-to-use huge pages is actually HugePages_Free minus HugePages_Rsvd.

Reference

1. Linux下hugetlbpage使用详解

PID

In POSIX, process is an instance of a running program and it contains one or more threads. The id of a process is called pid.

In the Linux kernel, task is the basic execution unit and is thread in the POSIX definition. Task is represented by struct task_struct in the code:

1
2
3
4
5
6

struct task_struct {
  ...
  pid_t   pid;
  pid_t   tgid;
  ...
}

Here, pid is the id of a task and tgid is the id of the thread group that contains the task. As we can see, task_struct.pid basically is POSIX thread id and task_struct.tgid is POSIX process id.

EXAMPLES

Two different meanings of pid can cause lots of confusions and it’s important to know whether we are talking about pid in the POSIX context or in the Linux kernel context.

getpid

getpid() is a function defined by the POSIX standard and pid means the id of a process in this context. Linux implements it by returning task_struct.tgid:

1
2
3

SYSCALL_DEFINE0(getpid) {
  return task_tgid_vnr(current);
}

/proc/[pid]

The proc file system is an interface to kernel data structures and pid means the id of a task/thread in this context. For example, /proc/[pid]/status shows status information about the task/thread and the implementation is in fs/proc/array.c.

References

[1] https://www.kernel.org/doc/ols/2002/ols2002-pages-330-337.pdf
[2] https://stackoverflow.com/questions/9305992/if-threads-share-the-same-pid-how-can-they-be-identified

Don’t directly use the performance numbers in this post, do your own measurement! Those numbers are highly dependent on the environment where those benchmarks are running.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

BENCHMARK_DEFINE_F(BenchmarkFixture, ForLoopAssignmentBenchmark)(benchmark::State& state) {
  for (auto _ : state) {
    char c[2048];
    for (size_t i = 0; i < 2048; ++i) {
      c[i] = 'a';
    }
    benchmark::DoNotOptimize(c);
  }
}
BENCHMARK_REGISTER_F(BenchmarkFixture, ForLoopAssignmentBenchmark);

BENCHMARK_DEFINE_F(BenchmarkFixture, MemsetBenchmark)(benchmark::State& state) {
  for (auto _ : state) {
    char c[2048];
    memset(c, 'a', 2048);
    benchmark::DoNotOptimize(c);
  }
}
BENCHMARK_REGISTER_F(BenchmarkFixture, MemsetBenchmark);

1
2
3
4
5
6

Run on (12 X 2500 MHz CPU s)
------------------------------------------------------------------------------------
Benchmark                                              Time           CPU Iterations
------------------------------------------------------------------------------------
BenchmarkFixture/ForLoopAssignmentBenchmark         1013 ns       1013 ns     750073
BenchmarkFixture/MemsetBenchmark                      84 ns         84 ns    6209926

As we can see, memset is much faster than the for loop assignment in this case. Looking at the generated assembly code, memset uses rep stos instruction which can be the reason why it’s faster.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

// https://shipilev.net/blog/2014/nanotrusting-nanotime/
inline uint64_t NanosecondsSinceEpoch() {
  timespec tp;
  clock_gettime(CLOCK_REALTIME, &tp);
  return tp.tv_sec * NS_PER_SEC + tp.tv_nsec;
}

BENCHMARK_DEFINE_F(BenchmarkFixture, NanoTimeLatencyBenchmark)(benchmark::State& state) {
  for (auto _ : state) {
    benchmark::DoNotOptimize(NanosecondsSinceEpoch());
  }
}
BENCHMARK_REGISTER_F(BenchmarkFixture, NanoTimeLatencyBenchmark);

BENCHMARK_DEFINE_F(BenchmarkFixture, NanoTimeGranularityBenchmark)(benchmark::State& state) {
  uint64_t cur_nano = 0;
  uint64_t last_nano = 0;
  for (auto _ : state) {
    do {
      cur_nano = NanosecondsSinceEpoch();
    } while (cur_nano == last_nano);
    last_nano = cur_nano;
  }
}
BENCHMARK_REGISTER_F(BenchmarkFixture, NanoTimeGranularityBenchmark);

1
2
3
4
5
6

Run on (12 X 2500 MHz CPU s)
------------------------------------------------------------------------------------
Benchmark                                              Time           CPU Iterations
------------------------------------------------------------------------------------
BenchmarkFixture/NanoTimeLatencyBenchmark             39 ns         39 ns   18361529
BenchmarkFixture/NanoTimeGranularityBenchmark         39 ns         39 ns   18073581

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

public class NanoTimeBenchmark {
  @State(Scope.Benchmark)
  public static class BenchmarkState {
    public long _lastNano;
    @Setup(Level.Iteration)
    public void setup() {
      _lastNano = 0;
    }
  }

  @Benchmark
  @Fork(1)
  @Threads(1)
  @Warmup(iterations = 10, time = 5, timeUnit = TimeUnit.SECONDS)
  @Measurement(iterations = 10, time = 5, timeUnit = TimeUnit.SECONDS)
  @BenchmarkMode(Mode.AverageTime)
  @OutputTimeUnit(TimeUnit.NANOSECONDS)
  public long nanoTimeLatency() {
    return System.nanoTime();
  }

  @Benchmark
  @Fork(1)
  @Threads(1)
  @Warmup(iterations = 10, time = 5, timeUnit = TimeUnit.SECONDS)
  @Measurement(iterations = 10, time = 5, timeUnit = TimeUnit.SECONDS)
  @BenchmarkMode(Mode.AverageTime)
  @OutputTimeUnit(TimeUnit.NANOSECONDS)
  public long nanoTimeGranularity(BenchmarkState benchmarkState) {
    long cur;
    do {
      cur = System.nanoTime();
    } while (cur == benchmarkState._lastNano);
    benchmarkState._lastNano = cur;
    return cur;
  }
}

1
2
3

Benchmark                              Mode  Cnt   Score   Error  Units
NanoTimeBenchmark.nanoTimeLatency      avgt   10  42.705 ± 0.302  ns/op
NanoTimeBenchmark.nanoTimeGranularity  avgt   10  43.875 ± 0.674  ns/op

This shows the overhead of getting time in both C++ and Java. They are not free!