I sat down this morning to write a straightforward post about automating customer service. The plan was the obvious one: here are the jobs a bot handles well, here are the jobs it does not, here is how you tell the difference. I have had that opinion for about two years and I was going to write it down.
Then I went to check something in POPIA, and found that the line I had been drawing by feel is already drawn in the Act.
Section 71
The heading is "Automated decision making". It says a person may not be subject to a decision that carries legal consequences for them, or affects them to a substantial degree, where that decision is based solely on automated processing intended to build a profile of them. The Act then lists what counts as a profile: performance at work, credit worthiness, reliability, location, health, personal preferences, conduct.
I had to read it twice. Not because it is complicated, but because it is the same list I would have written from experience, and I would have presented it as my judgement.
The part I could not have arrived at on my own is subsection (3). Where you rely on the exception and put "appropriate measures" in place, those measures have to do two specific things: give the person a chance to make representations about the decision, and give them enough information about the underlying logic of the automated processing to actually make those representations.
That second one is a design constraint with teeth. If a system declines somebody, they can ask why, and "the system decided" does not discharge the obligation. I have built things where that answer would have been the only one available.
What that changed about the post
The post I published this morning is not the one I sat down to write. It leads with the two statutes instead of with my opinion, because the statutes are better than my opinion and they were there the whole time. It is live at nexbdm.co.za, and the second half is still the practical part, which the law does not cover and should not.
The other Act in it is ECTA, from 2002, which calls an automated system an "electronic agent" and is thoroughly unsentimental about what happens when yours agrees to something. You are presumed bound by what it concluded, whether or not anyone at your business read it. The customer is only bound if a person could have reviewed the terms first. And if your flow gave them no way to catch a material mistake, there may be no agreement at all.
Twenty-four years old, written before any of this looked like it does now, and it lands squarely on a WhatsApp assistant that confirms a booking.
The thing I keep relearning
Ten months ago I would have said reading legislation was the least productive way to spend a morning. I wrote about stopping feature work to go and read the law when that turned around, and again when ten compliance guides in, the pieces turned out to be one chain.
This is a third version of it, and the useful distinction is that the first two were about compliance work. Today was not. Today was a product question, and I was answering it from taste.
Taste is not worthless. Most of what I decide in a week is taste, because most of it is not written down anywhere. But when there is a source, taste is the expensive way to get to a worse version of the same answer. The correction to make is not "read more law". It is: before I write down an opinion I have carried for years, check whether someone has already settled it.
I have been saying for a while that going to the primary document is the thing that separates our posts from the rest. I have been applying it to statistics and news claims. It had not occurred to me to apply it to my own opinions.
For anyone building something similar
Two practical things, both from this morning.
The first is that South Africa is better covered than people assume. There is a persistent belief that our law has nothing to say about automated systems and we are waiting for something to be written. Section 71 commenced on 1 July 2020, and the Act's one year transition ended on 1 July 2021, so it has been enforceable for over five years. ECTA has been there since 2002. Neither mentions AI, and both apply cleanly.
I nearly wrote "in force since July 2021" in that sentence, which is the compliance deadline rather than the commencement date. I checked it because it was a date, and dates are the kind of thing I have been wrong about before. It is a small correction and nobody would have caught it. That is rather the point of doing it anyway.
The second is smaller and I nearly got it wrong. The government gazette scan of ECTA is an image, and the text extraction off it is unreliable enough that one section number came out wrong. I went and found the wording in two other places before quoting any of it. On a post whose entire value is that the quotes are accurate, a plausible-looking OCR error would have been worse than not writing it.
That is the same discipline as checking the claim in the first place. It just applies one level further down, to the copy of the document you happen to be holding.