Fundraising Update 1

TL;DR: We’re pretty much on track, though we also haven’t hit the goal of pushing the fund past $78,890.69. Donate and help put the fund over the line!

With the short version out of the way, let’s dive into the details. What’s changed in the past week and change?

import datetime as dt

import matplotlib.pyplot as pl

import pandas as pd
import pandas.tseries.offsets as pdto

cutoff_day = dt.datetime( 2020, 5, 27, tzinfo=dt.timezone(dt.timedelta(hours=-6)) )

donations = pd.read_csv('donations.cleaned.tsv',sep='\t')

donations['epoch'] = pd.to_datetime(donations['created_at'])
donations['delta_epoch'] = donations['epoch'] - cutoff_day
donations['delta_epoch_days'] = donations['delta_epoch'].apply(lambda x: x.days)

# some adjustment is necessary to line up with the current total
donations['culm'] = donations['amount'].cumsum() + 14723

new_donations_mask = donations['delta_epoch_days'] > 0
print( f"There have been {sum(new_donations_mask)} donations since {cutoff_day}." )
There have been 8 donations since 2020-05-27 00:00:00-06:00.

There’s been a reasonable number of donations after I published that original post. What does that look like, relative to the previous graph?

pl.figure(num=None, figsize=(8, 4), dpi=150, facecolor='w', edgecolor='k')

pl.plot( donations['delta_epoch_days'], donations['culm'], '-',c='#aaaaaa')
pl.plot( donations['delta_epoch_days'][new_donations_mask], \
        donations['culm'][new_donations_mask], '-',c='#0099ff')

pl.title("Defense against Carrier SLAPP Suit")

pl.xlabel("days since cutoff")
pl.xlim( [-365.26,donations['delta_epoch_days'].max()] )
pl.ylim( [55000,82500] )

An updated chart from the past year. New donations are in blue.

That’s certainly an improvement in the short term, though the graph is much too zoomed out to say more. Let’s zoom in, and overlay the posterior.

# load the previously-fitted posterior
flat_chain = np.loadtxt('starting_posterior.csv')

pl.figure(num=None, figsize=(8, 4), dpi=150, facecolor='w', edgecolor='k')

x = np.array([0, donations['delta_epoch_days'].max()])
for m,_,_ in flat_chain:
    pl.plot( x, m*x + 78039, '-r', alpha=0.05 )
pl.plot( donations['delta_epoch_days'], donations['culm'], '-', c='#aaaaaa')
pl.plot( donations['delta_epoch_days'][new_donations_mask], \
        donations['culm'][new_donations_mask], '-', c='#0099ff')

pl.title("Defense against Carrier SLAPP Suit")

pl.xlabel("days since cutoff")
pl.xlim( [-3,x[1]+1] )
pl.ylim( [77800,79000] )

A zoomed-in view of the new donations, with posteriors overlaid.

Hmm, looks like we’re right where the posterior predicted we’d be. My targets were pretty modest, though, consisting of an increase of 3% and 10%, so this doesn’t mean they’ve been missed. Let’s extend the chart to day 16, and explicitly overlay the two targets I set out.

low_target = 78890.69
high_target = 78948.57
target_day = dt.datetime( 2020, 6, 12, 23, 59, tzinfo=dt.timezone(dt.timedelta(hours=-6)) )
target_since_cutoff = (target_day - cutoff_day).days

pl.figure(num=None, figsize=(8, 4), dpi=150, facecolor='w', edgecolor='k')

x = np.array([0, target_since_cutoff])
pl.fill_between( x, [78039, low_target], [78039, high_target], color='#ccbbbb', label='blog post')
pl.fill_between( x, [78039, high_target], [high_target, high_target], color='#ffeeee', label='video')

pl.plot( donations['delta_epoch_days'], donations['culm'], '-',c='#aaaaaa')
pl.plot( donations['delta_epoch_days'][new_donations_mask], \
        donations['culm'][new_donations_mask], '-',c='#0099ff')

pl.title("Defense against Carrier SLAPP Suit")

pl.xlabel("days since cutoff")
pl.xlim( [-3, target_since_cutoff] )
pl.ylim( [77800,high_target] )

pl.legend(loc='lower right')

The previous graph, this time with targets overlaid.

To earn a blog post and video on Bayes from me, we need the line to be in the pink zone by the time it reaches the end of the graph. For just the blog post, it need only be in the grayish- area. As you can see, it’s painfully close to being in line with the lower of two goals, though if nobody donates between now and Friday it’ll obviously fall quite short.

So if you want to see that blog post, get donating!

Fundraising Target Number 1

If our goal is to raise funds for a good cause, we should at least have an idea of where the funds are at.

(Click here to show the code)
created_at amount epoch delta_epoch culm
0 2017-01-24T07:27:51-06:00 10.0 2017-01-24 07:27:51-06:00 -1218 days +19:51:12 14733.0
1 2017-01-24T07:31:09-06:00 50.0 2017-01-24 07:31:09-06:00 -1218 days +19:54:30 14783.0
2 2017-01-24T07:41:20-06:00 100.0 2017-01-24 07:41:20-06:00 -1218 days +20:04:41 14883.0
3 2017-01-24T07:50:20-06:00 10.0 2017-01-24 07:50:20-06:00 -1218 days +20:13:41 14893.0
4 2017-01-24T08:03:26-06:00 25.0 2017-01-24 08:03:26-06:00 -1218 days +20:26:47 14918.0

Changing the dataset so the last donation happens at time zero makes it both easier to fit the data and easier to understand what’s happening. The first day after the last donation is now day one.

Donations from 2017 don’t tell us much about the current state of the fund, though, so let’s focus on just the last year.

(Click here to show the code)

The last year of donations, for the lawsuit fundraiser.

The donations seem to arrive in bursts, but there have been two quiet portions. One is thanks to the current pandemic, and the other was during last year’s late spring/early summer. It’s hard to tell what the donation rate is just by eye-ball, though. We need to smooth this out via a model.
The simplest such model is linear regression, aka. fitting a line. We want to incorporate uncertainty into the mix, which means a Bayesian fit. Now, what MCMC engine to use, hmmm…. emcee is my overall favourite, but I’m much too reliant on it. I’ve used PyMC3 a few times with success, but recently it’s been acting flaky. Time to pull out the big guns: Stan. I’ve been avoiding it because pystan‘s compilation times drove me nuts, but all the cool kids have switched to cmdstanpy when I looked away. Let’s give that a whirl.

(Click here to show the code)
CPU times: user 5.33 ms, sys: 7.33 ms, total: 12.7 ms
Wall time: 421 ms
CmdStan installed.

We can’t fit to the entire three-year time sequence, that just wouldn’t be fair given the recent slump in donations. How about the last six months? That covers both a few donation burts and a flat period, so it’s more in line with what we’d expect in future.

(Click here to show the code)
There were 117 donations over the last six months.

With the data prepped, we can shift to building the linear model.

(Click here to show the code)

I could have just gone with Stan’s basic model, but flat priors aren’t my style. My preferred prior for the slope is the inverse tangent, as it compensates for the tendency of large slope values to “bunch up” on one another. Stan doesn’t offer it by default, but the Cauchy distribution isn’t too far off.

We’d like the standard deviation to skew towards smaller values. It naturally tends to minimize itself when maximizing the likelihood, but an explicit skew will encourage this process along. Gelman and the Stan crew are drifting towards normal priors, but I still like a Cauchy prior for its weird properties.

Normally I’d plunk the Gaussian distribution in to handle divergence from the deterministic model, but I hear using Student’s T instead will cut down the influence of outliers. Thomas Wiecki recommends one degree of freedom, but Gelman and co. find that it leads to poor convergence in some cases. They recommend somewhere between three and seven degrees of freedom, but skew towards three, so I’ll go with the flow here.

The y-intercept could land pretty much anywhere, making its prior difficult to figure out. Yes, I’ve adjusted the time axis so that the last donation is at time zero, but the recent flat portion pretty much guarantees the y-intercept will be higher than the current amount of funds. The traditional approach is to use a flat prior for the intercept, and I can’t think of a good reason to ditch that.

Not convinced I picked good priors? That’s cool, there should be enough data here that the priors have minimal influence anyway. Moving on, let’s see how long compilation takes.

(Click here to show the code)
CPU times: user 4.91 ms, sys: 5.3 ms, total: 10.2 ms
Wall time: 20.2 s

This is one area where emcee really shines: as a pure python library, it has zero compilation time. Both PyMC3 and Stan need some time to fire up an external compiler, which adds overhead. Twenty seconds isn’t too bad, though, especially if it leads to quick sampling times.

(Click here to show the code)
CPU times: user 14.7 ms, sys: 24.7 ms, total: 39.4 ms
Wall time: 829 ms

And it does! emcee can be pretty zippy for a simple linear regression, but Stan is in another class altogether. PyMC3 floats somewhere between the two, in my experience.

Another great feature of Stan are the built-in diagnostics. These are really handy for confirming the posterior converged, and if not it can give you tips on what’s wrong with the model.

(Click here to show the code)
Processing csv files: /tmp/tmpyfx91ua9/linear_regression-202005262238-1-e393mc6t.csv, /tmp/tmpyfx91ua9/linear_regression-202005262238-2-8u_r8umk.csv, /tmp/tmpyfx91ua9/linear_regression-202005262238-3-m36dbylo.csv, /tmp/tmpyfx91ua9/linear_regression-202005262238-4-hxjnszfe.csv

Checking sampler transitions treedepth.
Treedepth satisfactory for all transitions.

Checking sampler transitions for divergences.
No divergent transitions found.

Checking E-BFMI - sampler transitions HMC potential energy.
E-BFMI satisfactory for all transitions.

Effective sample size satisfactory.

Split R-hat values satisfactory all parameters.

Processing complete, no problems detected.

The odds of a simple model with plenty of datapoints going sideways are pretty small, so this is another non-surprise. Enough waiting, though, let’s see the fit in action. First, we need to extract the posterior from the stored variables …

(Click here to show the code)
There are 256 samples in the posterior.

… and now free of its prison, we can plot the posterior against the original data. I’ll narrow the time window slightly, to make it easier to focus on the fit.

(Click here to show the code)

The same graph as before, but now slightly zoomed in on and with trendlines visible.

Looks like a decent fit to me, so we can start using it to answer a few questions. How much money is flowing into the fund each day, on average? How many years will it be until all those legal bills are paid off? Since humans aren’t good at counting in years, let’s also translate that number into a specific date.

(Click here to show the code)
mean/std/median slope = $51.62/1.65/51.76 per day

mean/std/median years to pay off the legal fees, relative to 2020-05-25 12:36:39-05:00 =

mean/median estimate for paying off debt =
	2022-05-12 07:49:55.274942-05:00 / 2022-05-09 13:57:13.461426-05:00

Mid-May 2022, eh? That’s… not ideal. How much time can we shave off, if we increase the donation rate? Let’s play out a few scenarios.

(Click here to show the code)
median estimate for paying off debt, increasing rate by   1% = 2022-05-02 17:16:37.476652800
median estimate for paying off debt, increasing rate by   3% = 2022-04-18 23:48:28.185868800
median estimate for paying off debt, increasing rate by  10% = 2022-03-05 21:00:48.510403200
median estimate for paying off debt, increasing rate by  30% = 2021-11-26 00:10:56.277984
median estimate for paying off debt, increasing rate by 100% = 2021-05-17 18:16:56.230752

Bumping up the donation rate by one percent is pitiful. A three percent increase will almost shave off a month, which is just barely worthwhile, and a ten percent increase will roll the date forward by two. Those sound like good starting points, so let’s make them official: increase the current donation rate by three percent, and I’ll start pumping out the aforementioned blog posts on Bayesian statistics. Manage to increase it by 10%, and I’ll also record them as videos.

As implied, I don’t intend to keep the same rate throughout this entire process. If you surprise me with your generosity, I’ll bump up the rate. By the same token, though, if we go through a dry spell I’ll decrease the rate so the targets are easier to hit. My goal is to have at least a 50% success rate on that lower bar. Wouldn’t that make it impossible to hit the video target? Remember, though, it’ll take some time to determine the success rate. That lag should make it possible to blow past the target, and by the time this becomes an issue I’ll have thought of a better fix.

Ah, but over what timeframe should this rate increase? We could easily blow past the three percent target if someone donates a hundred bucks tomorrow, after all, and it’s no fair to announce this and hope your wallets are ready to go in an instant. How about… sixteen days. You’ve got sixteen days to hit one of those rate targets. That’s a nice round number, for a computer scientist, and it should (hopefully!) give me just enough time to whip up the first post. What does that goal translate to, in absolute numbers?

(Click here to show the code)
a   3% increase over 16 days translates to $851.69 + $78039.00 = $78890.69

Right, if you want those blog posts to start flowing you’ve got to get that fundraiser total to $78,890.69 before June 12th. As for the video…

(Click here to show the code)
a  10% increase over 16 days translates to $909.57 + $78039.00 = $78948.57

… you’ve got to hit $78,948.57 by the same date.

Ready? Set? Get donating!

Deep Penetration Tests

We now live in an age where someone can back door your back door.

Analysts believe there are currently on the order of 10 billions Internet of Things (IoT) devices out in the wild. Sometimes, these devices find their way up people’s butts: as it turns out, cheap and low-power radio-connected chips aren’t just great for home automation – they’re also changing the way we interact with sex toys. In this talk, we’ll dive into the world of teledildonics and see how connected buttplugs’ security holds up against a vaguely motivated attacker, finding and exploiting vulnerabilities at every level of the stack, ultimately allowing us to compromise these toys and the devices they connect to.

Writing about this topic is hard, and not just because penises may be involved. IoT devices pose a grave security risk for all of us, but probably not for you personally. For instance, security cameras have been used to launch attacks on websites. When was the last time you updated the firmware on your security camera, or ran a security scan of it? Probably never. Has your security camera been taken over? Maybe, as of 2017 roughly half the internet-connected cameras in the USA were part of a botnet. Has it been hacked and commanded to send your data to a third party? Almost certainly not, these security cam hacks almost all target something else. Human beings are terrible at assessing risk in general, and the combination of catastrophic consequences to some people but minimal consequences to you only amplifies our weaknesses.

There’s a very fine line between “your car can be hacked to cause a crash!” and “some cars can be hacked to cause a crash,” between “your TV is tracking your viewing habits” and “your viewing habits are available to anyone who knows where to look!” Finding the right balance between complacency and alarmism is impossible given how much we don’t know. And as computers become more intertwined with our intimate lives, whole new incentives come into play. Proportionately, more people would be willing to file a police report about someone hacking their toaster than about someone hacking their butt plug. Not many people own a smart sex toy, but those that do form a very attractive hacking target.

There’s not much we can do about this individually. Forcing people to take an extensive course in internet security just to purchase a butt plug is blaming the victim, and asking the market to solve the problem doesn’t work when market incentives caused the problem in the first place. A proper solution requires collective action as a society, via laws and incentives that help protect our privacy.

Then, and only then, can you purchase your sex toys in peace.

The Crisis of the Mediocre Man

I was browsing YouTube videos on PyMC3, as one naturally does, when I happened to stumble on this gem.

Tech has spent millions of dollars in efforts to diversify workplaces. Despite this, it seems after each spell of progress, a series of retrograde events ensue. Anti-diversity manifestos, backlash to assertive hiring, and sexual misconduct scandals crop up every few months, sucking the air from every board room. This will be a digest of research, recent events, and pointers on women in STEM.

Lorena A. Barba really knows her stuff; the entire talk is a rapid-fire accounting of claims and counterclaims, aimed to directly appeal to the male techbros who need to hear it. There was a lot of new material in there, for me at least. I thought the only well-described matriarchies came from the African continent, but it turns out the Algonquin also fit that bill. Some digging turns up a rich mix of gender roles within First Nations peoples, most notably the Iroquois and Hopi. I was also depressed to hear that the R data analysis community is better at dealing with sexual harassment than the skeptic/atheist community.

But what really grabbed my ears was the section on gender quotas. I’ve long been a fan of them on logical grounds: if we truly believe the sexes are equal, then if we see unequal representation we know discrimination is happening. By forcing equality, we greatly reduce network effects where one gender can team up against the other. Worried about an increase in mediocrity? At worst that’s a temporary thing that disappears once the disadvantaged sex gets more experience, and at best the overall quality will actually go up. The research on quotas has advanced quite a bit since that old Skepchick post. Emphasis mine.

In 1993, Sweden’s Social Democratic Party centrally adopted a gender quota and imposed it on all the local branches of that party (…). Although their primary aim was to improve the representation of women, proponents of the quota observed that the reform had an impact on the competence of men. Inger Segelström (the chair of Social Democratic Women in Sweden (S-Kvinnor), 1995–2003) made this point succinctly in a personal communication:

At the time, our party’s quota policy of mandatory alternation of male and female names on all party lists became informally known as the crisis of the mediocre man

We study the selection of municipal politicians in Sweden with regard to their competence, both theoretically and empirically. Moreover, we exploit the Social Democratic quota as a shock to municipal politics and ask how it altered the competence of that party’s elected politicians, men as well as women, and leaders as well as followers.

Besley, Timothy. “Gender Quotas and the Crisis of the Mediocre Man: Theory and Evidence from Sweden.” THE AMERICAN ECONOMIC REVIEW 107, no. 8 (2017): 39.

We can explain this with the benefit of hindsight: if men can rely on the “old boy’s network” to keep them in power, they can afford to slack off. If other sexes cannot, they have to fight to earn their place. These are all social effects, though; if no sex holds a monopoly on operational competence in reality, the net result is a handful of brilliant women among a sea of iffy men. Gender quotas severely limit the social effects, effectively kicking out the mediocre men to make way for average women, and thus increase the average competence.

As tidy as that picture is, it’s wrong in one crucial detail. Emphasis again mine.

These estimates show that the overall effect mainly reflects an improvement in the selection of men. The coefficient in column 4 means that a 10-percentage-point larger quota bite (just below the cross-sectional average for all municipalities) raised the proportion of competent men by 4.4 percentage points. Given an average of 50 percent competent politicians in the average municipality (by definition, from the normalization), this corresponds to a 9 percent increase in the share of competent men.

For women, we obtain a negative coefficient in the regression specification without municipality trends, but a positive coefficient with trends. In neither case, however, is the estimate significantly different from zero, suggesting that the quota neither raised nor cut the share of competent women. This is interesting in view of the meritocratic critique of gender quotas, namely that raising the share of women through a quota must necessarily come at the price of lower competence among women.

Increasing the number of women does not also increase the number of incompetent women. When you introduce a quota, apparently, everyone works harder to justify being there. The only people truly hurt by gender quotas are mediocre men who rely on the Peter Principle.

The like ratio for said talk. 47 likes, 55 dislikes, FYI.Alas, if that YouTube like ratio is any indication, there’s a lot of them out there.

Sexism Poisons Everything

That black hole image was something, wasn’t it? For a few days, we all managed to forget the train wreck that is modern politics and celebrate science in its purest form. Alas, for some people there was one problem with M87’s black hole.

Dr. Katie Bouman, in front of a stack of hard drives.

A woman was involved! Despite the evidence that Dr. Bouman played a crucial role or had the expertise, they instead decided Andrew Chael had done all the work and she was faking it.

So apparently some (I hope very few) people online are using the fact that I am the primary developer of the eht-imaging software library () to launch awful and sexist attacks on my colleague and friend Katie Bouman. Stop.

Our papers used three independent imaging software libraries (…). While I wrote much of the code for one of these pipelines, Katie was a huge contributor to the software; it would have never worked without her contributions and

the work of many others who wrote code, debugged, and figured out how to use the code on challenging EHT data. With a few others, Katie also developed the imaging framework that rigorously tested all three codes and shaped the entire paper ();

as a result, this is probably the most vetted image in the history of radio interferometry. I’m thrilled Katie is getting recognition for her work and that she’s inspiring people as an example of women’s leadership in STEM. I’m also thrilled she’s pointing

out that this was a team effort including contributions from many junior scientists, including many women junior scientists (). Together, we all make each other’s work better; the number of commits doesn’t tell the full story of who was indispensable.

Amusingly, their attempt to beat back social justice within the sciences kinda backfired.

As openly lesbian, gay, bisexual, transgender, queer, intersex, asexual, and other gender/sexual minority (LGBTQIA+) members of the astronomical community, we strongly believe that there is no place for discrimination based on sexual orientation/preference or gender identity/expression. We want to actively maintain and promote a safe, accepting and supportive environment in all our work places. We invite other LGBTQIA+ members of the astronomical community to join us in being visible and to reach out to those who still feel that it is not yet safe for them to be public.

As experts, TAs, instructors, professors and technical staff, we serve as professional role models every day. Let us also become positive examples of members of the LGBTQIA+ community at large.

We also invite everyone in our community, regardless how you identify yourself, to become an ally and make visible your acceptance of LGBTQIA+ people. We urge you to make visible (and audible) your objections to derogatory comments and “jokes” about LGBTQIA+ people.

In the light of the above statements, we, your fellow students, alumni/ae, faculty, coworkers, and friends, sign this message.

Andrew Chael, Graduate Student, Harvard-Smithsonian Center for Astrophysics

Yep, the poster boy for those anti-SJWs is an SJW himself!

So while I appreciate the congratulations on a result that I worked hard on for years, if you are congratulating me because you have a sexist vendetta against Katie, please go away and reconsider your priorities in life. Otherwise, stick around — I hope to start tweeting

more about black holes and other subjects I am passionate about — including space, being a gay astronomer, Ursula K. Le Guin, architecture, and musicals. Thanks for following me, and let me know if you have any questions about the EHT!

If you want a simple reason why I spend far more time talking about sexism than religion, this is it. What has done more harm to the world, religion or sexism? Which of the two depends most heavily on poor arguments and evidence? While religion can do good things once in a while, sexism is prevented from that by definition.

Nevermind religion, sexism poisons everything.

… Whoops, I should probably read Pharyngula more often. Ah well, my rant at the end was still worth the effort.

Ridiculously Complex

Things have gotten quiet over here, due to SIGGRAPH. Picture a giant box of computer graphics nerds, crossed with a shit-tonne of cash, and you get the basic idea. And the papers! A lot of it is complicated and math-heavy or detailing speculative hardware, sprinkled with the slightly strange. Some of it, though, is fairly accessible.

This panel on colour, in particular, was a treat. I’ve been fascinated by colour and visual perception for years, and was even lucky enough to do two lectures on the subject. It’s a ridiculously complicated subject! For instance, purple isn’t a real colour.

The visible spectrum of light. Copyright Spigget, CC-BY-SA-3.0.

Ok ok, it’s definitely “real” in the sense that you can have the sensation of it, but there is no single wavelength of light associated with it. To make the colour, you have to combine both red-ish and blue-ish light. That might seem strange; isn’t there a purple-ish section at the back of the rainbow labeled “violet?” Since all the colours of the rainbow are “real” in the single-wavelength sense, a red-blue single wavelength must be real too.

It turns out that’s all a trick of the eye. We detect colour through one of three cone-shaped photoreceptors, dubbed “long,” “medium,” and “short.” These vary in what sort of light they’re sensitive to, and overlap a surprising amount.

Figure 2, from Bowmaker & Dartnall 1980. Cone response curves have been colourized to approximately their peak colour response.

Your brain determines the colour by weighing the relative response of the cone cells. Light with a wavelength of 650 nanometres tickles the long cone far more than the medium one, and more still than the short cone, and we’ve labeled that colour “red.” With 440nm light, it’s now the short cone that blasts a signal while the medium and long cones are more reserved, so we slap “blue” on that.

Notice that when we get to 400nm light, our long cones start becoming more active, even as the short ones are less so and the medium ones aren’t doing much? Proportionately, the share of “red” is gaining on the “blue,” and our brain interprets that as a mixture of the two colours. Hence, “violet” has that red-blue sensation even though there’s no light arriving from the red end of the spectrum.

To make things even more confusing, your eye doesn’t fire those cone signals directly back to the brain. Instead, ganglions merge the “long” and “medium” signals together, firing faster if there’s more “long” than “medium” and vice-versa. That combined signal is itself combined with the “short” signal, firing faster if there’s more “long”/”medium” than “short.” Finally, all the cone and rod cells are merged, firing more if they’re brighter than nominal. Hence where there’s no such thing as a reddish-green nor a yellow-ish blue, because both would be interpreted as an absence of colour.

I could (and have!) go on for an hour or two, and yet barely scratch the surface of how we try to standardize what goes on in our heads. Thus why it was cool to see some experts in the field give their own introduction to colour representation at SIGGRAPH. I recommend tuning in.


A Little Racist Butterfly

Researchers have noted that, for decades, prison sentences have been just ever-so-slightly more harsh for black people than white people.

As a whole, these findings undermine the so-called ‘‘no discrimination thesis’’ which contends that once adequate controls for other factors, especially legal factors (i.e., criminal history and severity of current offense), are controlled unwarranted racial disparity disappears. In contrast to the no discrimination thesis, the current research found that independent of other measured factors, on average African-Americans were sentenced more harshly than whites. The observed differences between whites and African Americans generally were small, suggesting that discrimination in the sentencing stage is not the primary cause of the overrepresentation of African-Americans in U.S. correctional facilities.

Mitchell, Ojmarrh. “A meta-analysis of race and sentencing research: Explaining the inconsistencies.” Journal of Quantitative Criminology 21.4 (2005): 439-466.

Not as widely noted: incarceration sorta behaves like a contagious disease. [Read more…]

Continued Fractions

If you’ve followed my work for a while, you’ve probably noted my love of low-discrepancy sequences. Any time I want to do a uniform sample, and I’m not sure when I’ll stop, I’ll reach for an additive recurrence: repeatedly sum an irrational number with itself, check if the sum is bigger than one, and if so chop it down. Dirt easy, super-fast, and most of the time it gives great results.

But finding the best irrational numbers to add has been a bit of a juggle. The Wikipedia page recommends primes, but it also claimed this was the best choice of all:\frac{\sqrt{5} - 1}{2}

I couldn’t see why. I made a half-hearted attempt at digging through the references, but it got too complicated for me and I was more focused on the results, anyway. So I quickly shelved that and returned to just trusting that they worked.

That is, until this Numberphile video explained them with crystal clarity. Not getting the connection? The worst possible number to use in an additive recurrence is a rational number: it’ll start repeating earlier points and you’ll miss at least half the numbers you could have used. This is precisely like having outward spokes on your flower (no seriously, watch the video), and so you’re also looking for any irrational number that’s poorly approximated by any rational number. And, wouldn’t you know it…

\frac{\sqrt{5} - 1}{2} ~=~ \frac{\sqrt{5} + 1}{2} - 1 ~=~ \phi - 1

… I’ve relied on the Golden Ratio without realising it.

Want to play around a bit with continued fractions? I whipped up a bit of Go which allows you to translate any number into the integer sequence behind its fraction. Go ahead, muck with the thing and see what patterns pop out.