Feb 5, 2022

AI Dermatology: Part 1

Midway through last year Google announced a new foray into the medical technology space, sharing that it was developing an “AI-powered dermatology assist tool”—a phone-based app that would allow users to take photos of skin lesions and retrieve information about relevant medical conditions from the web. Similar apps already exist, but it’s fair to say that a comparable effort by Google is likely to have much more significant effects on how people interact with the medical system, their personal data, and even their own bodies.

Tags:  machine-learning ,  dermatology ,  medicine ,  data ,  privacy ,  foundation-models

Dec 19, 2020

Vaccine Allocation at Stanford Hospital

In a video that was widely shared last Friday, a representative from Stanford Medical Center spoke to residents protesting how the hospital chose to allocate its first shipment of COVID-19 vaccines. The hospital had around 5,000 initial doses to distribute (and expects to have tens of thousands more within the next few weeks), and came up with an allocation scheme in which only 7 of the approximately 1,300 residents were on the list. Many of these residents deal directly with patients who have COVID-19, whereas other more senior physicians, as well as other front line workers, such as nurses and food service employees, were given priority. In the video, the spokesperson explains that the algorithm they used to come up with an allocation scheme “clearly didn’t work”, to which protestors respond by shouting “Algorithms suck!” and “Fuck the algorithm!” …

Tags:  algorithmic-decision-making ,  medicine ,  communication ,  covid

Oct 3, 2020

The case for professional critics in science

In many areas of science, there is an increasingly urgent unmet need, a role that could be simultaneously fascinating, rewarding, and potential remunerative. It is a role that already exists in various forms, but which could be made into something much more potent, especially if forces converged to make it more prominent. I am talking, of course, about the professional science critic.

In the popular imagination, science operates something like a priesthood: scientists enter elite institutions as novices and emerge years later as full-fledged representatives of The Truth. Along the way, they are trained to be experts and professionals in their subject, to excel in action as well as in thought, and to sacrifice their worldly interests for the sake of their calling. The scholarly journals and peer review process are imagined to operate like an ecumenical council, guiding and filtering the thoughts of the broader community of the devout, and only allowing to pass what is deemed to be true and useful and good.

Tags:  science ,  criticism

Jul 20, 2019

Representational Power

The original plate of “View from the Window at Le Gras”, a heliograph made by Nicéphore Niépce around 1827.

The original plate of “View from the Window at Le Gras”, a heliograph made by Nicéphore Niépce around 1827.

Although it isn’t normally thought of in these terms, taking a photograph involves recording a four-dimensional block of space-time and projecting it down to a two-dimensional representation. With sufficiently sensitive material (and a fast enough shutter), one can produce images more or less instantaneously, but longer exposures reveal the inherent temporality of this process, showing us something that is clearly based on the world, yet quite different from our experience of it. Today, the ability to create images is so commonplace, of course, that we easily take it for granted, but early commentaries on photography reveal just how extraordinary it once was. Indeed, the history of photography provides both a compelling example of the power of representation, and a useful parallel to more recent forms of technological magic, especially that of machine learning.

Tags:  photography ,  representation ,  machine-learning ,  power

Dec 17, 2018

On the Perils of Automated Face Recognition

Anthropometric data sheet (both sides) of Alphonse Bertillon (1853–1914).

Anthropometric data sheet (both sides) of Alphonse Bertillon (1853–1914).

For anyone who has been paying attention, it will not have gone unnoticed that the past year has seen a dramatic expansion in the use of face recognition technology, including at schools, border crossing, and interactions with the police. Most recently, Delta announced that some passengers in Atlanta will be able to check in and go through security using only their face as identification. Most news coverage of this announcement emphasized the supposed convenience, efficiency, and technical novelty, while underplaying any potential hazards. In fact, however, the combination of widely available images, the ability to build on existing infrastructure, and a legal landscape that places very few restrictions on recording, means that face recognition represents a unique threat to privacy that should concern us greatly.

Tags:  privacy ,  face-recognition ,  machine-learning ,  surveillance ,  biometrics

May 25, 2018

What everyone needs to know about interpretability in machine learning

For anyone who’s been paying attention, it should be apparent that statistical machine learning systems are being widely deployed for automated decision making in all kinds of areas these days, including criminal justice, medicine, education, employment, policing, and so on. Particularly with the recently enacted GDPR—the new European regulation about data and privacy—there is growing interest in having systems that are interpretable, that is, we can make some sense of why they are making the prediction that they are making. To borrow an example from Been Kim, if a computer tells you that you need surgery, you’re probably going to ask for some sort of explanation.

Tags:  machine-learning ,  interpretability

May 14, 2018

Privacy in Context

Although it was not the largest of its kind, or the most invasive, or even particularly surprising, the recent Cambridge Analytica scandal produced a surprisingly large amount of outrage and commentary. If nothing else, it was yet another reminder that we have gradually slipped into a regime where certain aspects of our privacy that could once be taken for granted are now long gone. Are people concerned? Is this something we should be worried about? What exactly are the harms that come from this sort of loss of privacy.

Tags:  privacy ,  books

Nov 27, 2017

Emergent Archives

It’s fascinating to think about how much we give to the internet, and sometimes, how much it gives back. In my mind, there is a fairly direct historical connection between self-archiving practices from a couple of decades ago (keeping a diary, making photo albums, writing letters, etc.), to all the more recent variations on this idea (status updates, tweets, photo feeds, etc.). The difference is arguably in how much of the background work is taken care of for us, and the fact that we now make so much of this information public or semi-public. While many have explored the implications of this for privacy, security, and surveillance, there is another aspect that gets less attention: the automatic creation of our own archives.

Tags:  archives ,  memory

Nov 25, 2017

Montage as Archive

I was recently reminded of Christian Marclay’s The Clock—a 24 hour film made up of thousands of short film clips, carefully edited together such that (when properly synchronized) the time depicted in each moment of film corresponds to the current time in the world. I’ve never seen it, and sadly only a few copies exist, but I think it’s a fascinating example of a kind of archival art.

When you hear the description, you might think of it as just a kind of a gimmick, one which would not reward actual viewing. However, based on the (bootlegged) clips that are available online and reports from various people who have seen it, it apparently achieves much more. To be clear, although many of the scenes involve a shot of some sort of time piece (thereby establishing the time in the source film), the composition is not so simple as just finding and assembling shots of clocks. Rather, we see a slice in time, sometimes with intercutting between films. As a result, common patterns quickly emerge: in the morning we get people waking up. In the evening, we see people eating dinner and attending parties, etc. This is interesting because what is shown in films bears some relation to what happens in the real world; even if it ends up being a distorted or idealized reflection, this certainly tells us something about how Hollywood depicts society, and by implication, something about society itself.

Tags:  archives ,  film ,  history

Jun 5, 2017

Reproducibility in Art and Science

Marking the beginning of a new partnership between Rhizome and the Google Cultural Institute, the Rhizome blog recently published a conversation between Dragan Espenschied, Rhizome’s preservation director, and Vint Cerf, Google’s Chief Internet Evangelist. In addition to bringing together two people with among the coolest job titles ever, it got me thinking about some similarities between art and science when it comes to the issue of reproduction.

The main thrust of the conversation is about the difficulty of preserving internet art. In particular, works such as The Web Stalker were often made in a particular context, intended to be performances of a sort, and which depended on the existence of a certain technical infrastructure, including a particular operating system, a particular input device, an internet connection, and accompanying protocols. Part of the conversation relates to the fact that it’s actually very difficult to maintain or recreate every single thing that is required in order for a modern viewer to have the exact same experience as when it was first created. Nevertheless, Dragan argues, there is still value in preserving part of the experience.

Tags:  reproducibility ,  art ,  science ,  digital-preservation ,  open-science