Aaron Diaz, author of the webcomic Dresden Codak (one of the most scientifically and philosophically literate webcomics on the internet) and “Enough is Enough: a Thinking Ape’s Critique of Trans-Simianism”, a hilarious defense of transhumanism, has now written “Artificial Flight and Other Myths (a reasoned examination of A.F. by top birds)”, which pokes fun at those who think that Artificial Intelligence will require replicating every aspect of the human brain. Here is the opening:

Artificial Flight and Other Myths
a reasoned examination of A.F. by top birds

Over the past sixty years, our most impressive developments have undoubtedly been within the industry of automation, and many of our fellow birds believe the next inevitable step will involve significant advancements in the field of Artificial Flight. While residing currently in the realm of science fiction, true powered, artificial flying mechanisms may be a reality within fifty years. Or so the futurists would have us believe. Despite the current media buzz surrounding the prospect of A.F., a critical examination of even the most basic facts can dismiss the notion of true artificial flight as not much more than fantasy.

We can start with a loose definition of flight. While no two bird scientists or philosophers can agree on the specifics, there is still a common, intuitive understanding of what true flight is: powered, feathered locomotion through the air through the use of flapping wings. While other flight-like phenomena exist in nature (via bats and insects), no bird with even a reasonable education would consider these creatures true fliers, as they lack one or more key elements. And, while some birds are unfortunately born handicapped (penguins, ostriches, etc.), they still possess the (albeit undeveloped) gene for flight, and it is indeed flight that defines the modern bird.

This is flight in the natural world, the product of millions of years of evolution, and not a phenomenon easily replicated. Current A.F. is limited to unpowered gliding; a technical marvel, but nowhere near the sophistication of a bird. Gliding simplifies our lives, and no bird (including myself) would discourage advancing this field, but it is a far cry from synthesizing the millions of cells within the wing alone to achieve Strong A.F. Strong A.F., as it is defined by researchers, is any artificial flier that is capable of passing the Tern Test (developed by A.F. pioneer Alan Tern), which involves convincing an average bird that the artificial flier is in fact a flying bird.

Continue here.

g-speak overview 1828121108 from john underkoffler on Vimeo.

Here is the NYT article.

My understanding of the concept of anthropomorphism really “clicked” when I first read “Beyond anthropomorphism”, part of Creating Friendly AI, an early (2000) Singularity Institute document. I strongly recommend it for those who are interested in better understanding the concept of non-anthropomorphic artificial intelligence. Here is the opening:

If you punch a human in the nose, he or she will punch back. If the human doesn’t punch back, it’s an admirable act of self-restraint, something worthy of note.

Imagine, for a moment, that you walk up and punch an AI in the nose. Does the AI punch back? Perhaps and perhaps not, but punching back will not be instinctive. A sufficiently young AI might stand there and think: “Hm. Someone’s fist just bumped into my nose.” In a punched human, blood races, adrenaline pumps, the hands form fists, the stance changes, all without conscious attention. For a young AI, focus of attention shifts in response to an unexpected negative event - and that’s all.

As the AI thinks about the fist that bumped into vis nose, it may occur to the AI that this experience may be a repeatable event rather than a one-time event, and since a punch is a negative event, it may be worth thinking about how to prevent future punches, or soften the negativity. An infant AI - one that hasn’t learned about social concepts yet - will probably think something like: “Hm. A fist just hit my nose. I’d better not stand here next time.”

The more I study nature and biology, the more I see that anthropomorphism gets in the way of understanding animals as well. Certain birds, cats, dogs, and even rodents are intelligent, but thinking of their intelligence merely as inferior to humans is not the whole story. Different forms of intelligence have to be understood on their own terms — not through starting with an archetype of human intelligence and making incremental modifications to that archetype. That sort of thinking can lead to anchoring.

Tom McCabe at the Rational Futurist has a new article up; “The Real Story Behind Fusion Energy”. I suggest you check it out — it dispels a great many myths that we have been told about fusion power, and recommends the construction of thorium-powered fission reactors instead.

Meat and egg companies often try to sell their wares to unsuspecting SWPLs (”socially conscious” educated bourgeoisie Americans) by using the “free range” label. Unsurprisingly, this label is a lie. To quote the Wikipedia page on “free range”:

The U.S. Department of Agriculture Food Safety and Inspection Service (FSIS) requires that chickens raised for their meat have access to the outside in order to receive the free-range certification. There is no requirement for access to pasture, and there may be access to only dirt or gravel. Free-range chicken eggs, however, have no legal definition in the United States. Likewise, free-range egg producers have no common standard on what the term means. Many egg farmers sell their eggs as free range merely because their cages are two or three inches above average size, or because there is a window in the shed.

The USDA has no specific definition for “free-range” beef, pork, and other non-poultry products. All USDA definitions of “free-range” refer specifically to poultry. No other criteria-such as the size of the range or the amount of space given to each animal-are required before beef, lamb, and pork can be called “free-range”. Claims and labeling using “free range” are therefore unregulated. The USDA relies “upon producer testimonials to support the accuracy of these claims.”

Basically, the label is a farce. It conjures up images of old time family farms, when the reality is the exact opposite. Factory farmed chickens are routinely debeaked, and starved to cause forced molting, which shocks them into entering another egg-laying cycle. They live in filthy, shit-strewn cages and suffer from respiratory diseases due to inhaling large quantities of nitrogen released by their feces. “Free range” chickens spend most of their time in cages.

Specifically regarding eggs, here’s another source, the Humane Society:

The vast number of consumer labels affixed to egg cartons can leave a shopper feeling dazed and confused. One carton may label its eggs “Natural.” Another carton may call them “Free Range,” while yet another may claim its eggs are “Certified Organic.” How are thoughtful consumers supposed to know what these labels and claims really mean?

The truth is that the majority of egg labels have little relevance to animal welfare or, if they do, they have no official standards or any mechanism to enforce them.

Here’s another, more detailed pro-animal rights source on the Free-Range Myth. This organization, the Peaceful Prarie Sanctuary, provides a safe haven for animals rescued from factory farming. After their miserable lives, “spent hens” are terminated immediately by the egg-laying operations themselves, as their meat has no market value. Easy methods of termination include gas chambers, woodchippers, or simply throwing them into a dumpster to die. In one case, an egg-laying operation was caught red-handed burying thousands of hens in large trenches because it was apparently too inconvenient to send them to the rendering plant. Anything to get the job done and home in time for dinner, you know?

Whether or not you care about the welfare of chickens, the misleading presentation that factory farmers use to sell their goods is designed to instill false beliefs in consumers, and it is a good case study in deception. People want to believe that cage-free actually means cage-free, so they can feel good, but the whole idea is merely a falsity perpetuated by gullible consumers and cynical ranch owners. Essentially, humans are completely comfortable inflicting the worst imaginable suffering on any number of pigs, cows, and chickens to satisfy our taste buds, yet we expect transhumans and posthumans to treat us with respect. Why? The tyrant that carelessly inflicts brutality on his subjects is liable to get his just desserts sooner or later.

Do understand that kindness to animals is not necessarily all-or-nothing. One person can have a tremendous impact simply by making an effort to lower the number of animal products they consume per week.

While I’m addressing the topic, I might as well point out that “what about the suffering of broccoli and other plants?” is one of the most intellectually pathetic comeback arguments I have ever heard to justify factory farming. Everyone knows that plants lack nerve cells, never mind brains. Only someone completely ignorant of the most basic biology could plausibly make sure an argument. The truth is that that argument is merely a pithy joke designed to mock pro-animal rights arguments through misdirection. Its common use only illustrates that a substantial number of people who consume animal products see no need to justify their actions, and make no pretense at devoting any thought to the issue.

My apologies, but I will leave comments off for this post, because 1) the most plausible comments are likely to be from people who regard animals as dirt and are just trying to eliminate guilt by providing a pithy comeback, and 2) I don’t want to start too much of a precedent for animal rights debates on this blog, because there are many other places around the Internet to have them, and as far as I am concerned, the “debate” is mostly a non-issue. Yes, perhaps I could “win some people over” by being polite and engaging them in the comments, but it doesn’t really matter, because I am extremely doubtful that anything less than in vitro meat will bring down factory farming. Factory farming operations are expanding at a massive rate as the world’s standard of living increases but its empathy for animals remains where it has been throughout most of history — in the toilet.

John Robb on Homemade Microwave Weapons
James Hughes: Problems of Transhumanism: Liberal Democracy vs. Technocratic Absolutism
Technology Review: Defining an Algorithm for Inventing from Nature
New Study: Human Running Speeds of 35 to 40 mph May be Biologically Possible
NASA’s Puffin: Will It Be the Personal Transport Vehicle of our SciFi Future?
Simon Conway Morris: Aliens are Likely to Look and Behave Like Us (What about the Zerg?)
Current TV’s Max and Jason on Connecting Science and Culture
Patrick Millard: Open Sim Project
Nick Bostrom: Moral Uncertainty: Towards a Solution?
Humanity+ Conference in London in April
Wired: Removing Part of Skull Makes for Better Brain Scans
Scientific American: Time to Ban Production of Nuclear Weapons Material
Ray Kurzweil at SU/MIT/X Prize BCI Workshop (More from Singularity Hub)
Gary Kasparov on AI: The Chess Master and the Computer
Nanowerk: Simple DNA Nanomachine is Capable of Continuous Rotation
Video Gamers: Size of Brain Structures Predicts Success
Robots Climb Up the Wall (w/ Video)
Retail Meat Linked to Urinary Tract Infections: Strong Evidence
The Human Brain Uses a Grid to Represent Space
Scientists Identify Ecuador’s Yasuní National Park as one of the Most Biodiverse Places on Earth
Face Recognition Ability Inherited Separately from IQ
Bill Gates’ New Website
Researchers Discover Ebola’s Deadly Secret
Study suggests theory for insect colonies as ’superorganisms’
Explained: the Shannon Limit
Wired: Never Mind the Singularity, Here’s the Science
Utopian Pessimist Calls on Radical Tech to Save Economy
A Lawyer’s View of the Risk of Black Hole Catastrophe at the LHC
Aubrey de Grey in Helsinki, Finland
Will the First Self-Replicating Machine Be Our Last Invention?

If you haven’t picked up this month’s GQ magazine, do it soon. There is a feature on the Singularity Summit and Singularity Institute. (I also hear there is a piece by Carl Zimmer on the Singularity in Playboy but I haven’t picked it up yet.) Seeing community names like Rick Schwall (an SIAI donor and supporter) in a national magazine sure is a trip. According to the National Magazine Awards, circulation is somewhere between 500,000 and 1,000,000 and is up in recent years. Here is the cover, in all its glory:

And here, I blew up the Singularity portion for emphasis:

Really freaky, mmhmmm! Freaky like our ancestral past or Pandora freaky, I hope.

In case you hadn’t heard, there is an article by Bill Gates up at Huffington Post, “Why We Need Innovation, Not Just Insulation”. Here’s how it starts:

People often present two timeframes that we should have as goals for CO2 reduction - 30% (off of some baseline) by 2025 and 80% by 2050.

I believe the key one to achieve is 80% by 2050.

But we tend to focus on the first one since it is much more concrete.

We don’t distinguish properly between things that put you on a path to making the 80% goal by 2050 and things that don’t really help.

Most people “concerned” about global warming are caught up in Gaianist nonsense, Al Gore-flavored uneducated alarmism, and eco-bling. They will think whatever a small cadre of politicians and elite academics want them to think.

Stewart Brand, thankfully, has been facing up to the truth that we need nuclear power to permanently lower carbon emissions. Jamais Cascio has been introducing geoengineering to the discussion, and it was recently reported that geoengineering research is being funded by Gates. More radically, J. Storrs Hall has proposed a weather machine which he claims could be built within a few decades.

Unfortunately, even if we ceased all carbon emissions tomorrow, the thermal inertia of the oceans will ensure that warming continues for “a century or more”. Of course, pointing this out at all is considered defeatist in many quarters, but too bad.

As I’ve always said, the easiest ways for people to fight global warming right now are halting meat consumption, traveling less, and moving into smaller houses. Al Gore could do much more to fight global warming if he pushed these lifestyle changes aggressively. Yet Gore keeps living in a big house, traveling all over the place, and eating meat. He sets a bad example and decreases the credibility of the movement as a whole. People concerned about global warming — please spare me your boring essays about the need to reduce emissions. I’m only interested in seeing your latest vegetarian recipes, pictures of your bicycle, and your small, well-insulated apartment. Show, don’t tell.

Here is the link. This is a good place to start to review Kurzweil’s 1996-1997 predictions. I remember reading this chapter myself in 2000 and analyzing the way in which the predictions did sync up with my own and the way they did not.

There are two categories of qualifying words used for the technology predictions: either they’re 1) “ubiquitous”, “common”, or the like, or 2) they simply exist. For something to qualify as “common” in my eyes would perhaps mean that a third of the white collar business world in the United States uses it on a weekly basis. (To be very generous.) For #2, the prediction can be regarded as having come “true” even if the product only exists as a prototype in a lab and has for some time.

Here’s a comment from Keith Norbury on the Kurzweil response post that I agree with:

It looks as though Kurzweil and Anissimov are both quibbling. I had similar thoughts as Anissimov did when I scrolled through the predictions in The Age of Spiritual Machines. But I also thought, well, Kurzweil is just a little hasty in his enthusiasm. Yes, there’s a danger in setting firm dates for predictions of technological progress. However, because he makes them, Kurzweil gets people’s attention. Even when he is wrong on the exact date, he is still able to point to a trend that indicates he will be right soon enough (in most cases). So far, though, the dates have passed for the easier predictions. It gets harder going ahead.

Kurzweil’s main point is that technology is improving exponentially not linearly. That’s a difficult point to grasp. However, we still don’t know if even exponential growth is enough to tackle some sticky problems, such as simulating human intelligence. Nobody knows where the goal posts are yet. Nor do we understand yet the principles involved in uploading a human mind to computer, never mind the engineering it would require. The answers might be just around the corner, or they might be a long way away. Time travel, for example, is possible under the laws of physics. However, the huge energies required pose a giant obstacle to making it a reality.

I’m now reading Kim Stanley Robinson’s excellent Red Mars, which points out the difficulties in making predictions. It’s speculative fiction but also hard science fiction. The trouble is, though, that the hard science in Red Mars is the science of 1993 when it was written. In the book, the voyage to Mars took nine months, as predicted using the technology that was proven in 1993. Since then, an ion propulsion system is well along the road to development that promises to shorten the trip to about 40 days — when it happens. That certainly doesn’t look like it will be by 2026, as in Red Mars. One could argue that Robinson wasn’t being a futurist when he wrote Red Mars. However, at the time he was striving to imagine as accurately as he could, based on the knowledge available, what that future mission would look like. Unfortunately, he didn’t imagine that humans would develop a better technology for getting to Mars, even though the principles of ion propulsion were already well known back in the 1990s.

My guess is that Robinson, in writing Red Mars, was thinking too linearly about technological progress and not in the exponential way that Kurzweil does. That’s what sets Kurzweil apart from other intelligent people who speculate about the future.

I agree with Kurzweil that many important technological metrics are improving exponentially, and that his linear-thinking critics are incorrect. I have always argued that major change is likely in the relatively near future. I regard a Singularity at 2029 or earlier as definitely within the realm of possibility. I am a “Singularitarian” of the type that Kurzweil describes in his book. Much of my life is focused around the idea of a Singularity, similar but not the same as Kurzweil’s idea of the Singularity. I object to Kurzweil’s statements that MNT and nanorobots will certainly be a reality in the 2020s. I object to a lot of other things. I agree on the broad outlines of exponential change. I do not think Kurzweil is an “idiot”, as Singularity Hub misleadingly claimed recently. I think Kurzweil is a genius and I applaud him for making predictions at all.

It is much easier to criticize than to make predictions, I admit that. I believe that Kurzweil’s model is a good framework, and my model of the future is extremely similar to his relative to the mainstream. Still, the fine points are worth arguing. My main focus is on the points themselves. Perhaps I should have just listed the items and not even called them Kurzweil’s predictions, so I could criticize them at will without in any way threatening his reputation. In any case, I don’t think that Kurzweil’s reputation is at risk here. As he pointed out, I just poked at 7 out of 108 of his predictions in the book. I apologize for the sensationalist title of my original post — I didn’t mean that ALL of Kurzweil’s predictions for 2009 had failed, just “Here’s a few failed predictions that I found on this specific web page and I agree with”.

I’m sure that everyone is interested in seeing Kurzweil’s point-by-point analysis of his predictions in The Age of Spiritual Machines. Considering the concerns raised by those seven predictions I mentioned, I think a thorough review of the book is in order, and I’m pleased that Kurzweil himself has taken up the task. I gave the original post a provocative title because I strongly believed that investigation would benefit the entire futurist community, and I hoped to start a conservation on it. In that respect, it appears to have succeeded.

In February 2009, the President of the American Association for Artificial Intelligence, Eric Horvitz, convened a panel on “long-term AI futures” which explicitly delved into issues around the Singularity and intelligence explosion. Horvitz has told me (and the New York Times) that the reason he convened the panel was not due to personal interest or concern in the issue but in response to the public interest and concern in the issue.

In the New York Times article covering the meeting, Horvitz was quoted as saying, “My sense was that sooner or later we would have to make some sort of statement or assessment, given the rising voice of the technorati and people very concerned about the rise of intelligent machines”. In August, they released an interim report that said:

Popular perspectives on the outcomes of AI research include expectation that there will be one or more disruptive outcomes. These include that notion that the research will somehow lead to the advent of utopia or catastrophe. The utopian perspective is perhaps best captured in the writings of Ray Kurzweil and others, who speak of a forthcoming “technological singularity.” At the other end of the spectrum, some people are concerned about the “rise of intelligent machines,” fueled by popular novels and movies, that tell stories of the loss of control of robots. Whether forecasting utopian or catastrophic outcomes, the radical perspectives are frightening to people in that they highlight some form of radical change on the horizon—often founded on a notion of the loss of control of the computational intelligences that we create.

The panel of experts was overall skeptical of the radical views expressed by futurists and science-fiction authors.

To me, this was a disappointing result. The phrasing is also disappointing. It is not just the opinion of “popular perspectives” that AI will “somehow” lead to the advent of utopia or catastrophe. Many academics (including AI researchers) have presented views that AI would be highly disruptive, including Ray Solomonoff, Nick Bostrom, Shane Legg, Matt Mahoney, I.J. Good, Bill Gates, Hans Moravec, Marvin Minsky, and many others. Solomonoff, Moravec, and Minsky have all been leaders in AI for decades, so it seems like a deliberate choice of focus to attribute “radical views” to the public rather than AI experts. It provides the AAAS panel with a comfortable level of removal from the claims, a level of removal they could not easily obtain if they cited Solomonoff, Moravec, and Minsky as the sources of Singularity views.

It is remarkable for the panel to suggest that AI will probably not result in disruptive outcomes — if you can turn a pile of sand into a thinking intelligence in the time it takes you to fabricate a computer chip and transfer files to it, then that wouldn’t be disruptive? In my view, it is the degree of disruption that is up for debate — I don’t take people very seriously if they imply there will be little or no disruption whatsoever.

In wondering why the panel came up with this result, Eliezer Yudkowsky suggested “snap consideration and snap judgment”. However, Steve Rayhawk offered a more detailed analysis, which I will post in its entirety here, with a few formatting changes to ensure successful reposting. The first two sentences are a quote that Rayhawk is responding to. Everything that follows from this point on (except for the last line and the quote) was posted by Steve Rayhawk to Less Wrong.

Roughly, what I expect to happen by default is no modular analysis at all - just snap consideration and snap judgment. I feel little need to explain such.

You, or somebody anyway, could still offer a modular causal model of that snap consideration and snap judgment. For example:

1. What cached models of the planning abilities of future machine intelligences did the academics have available when they made the snap judgment?
1.1 What fraction of the academics are aware of any current published AI architectures which could reliably reason over plans at the level of abstraction of “implement a proxy intelligence”?
1.1.1 What fraction of them have thought carefully about when there might be future practical AI architectures that could do this?
1.1.2 What fraction use a process for answering questions about the category distinctions that will be known in the future, which uses as an unconscious default the category distinctions known in the present?

2. What false claims have been made about AI in the past? What decision rules might academics have learned to use, to protect themselves from losing prestige for being associated with false claims like those?
2.1 How much do those decision rules refer to modular causal analyses of the object of a claim and of the fact that people are making the claim?
2.2 How much do those decision rules refer to intuitions about other peoples’ states of mind and social category memberships?
2.3 How much do those decision rules refer to intuitions about other peoples’ intuitive decision rules?
2.4 Historically, have peoples’ own abilities to do modular causal analyses been good enough to make them reliably safe from losing prestige by being associated with false claims? What fraction of academics have the intuitive impression that their own ability to do analysis isn’t good enough to make them reliably safe from losing prestige by association with a false claim, so that they can only be safe if they use intuitions about the states of mind and social category memberships of a claim’s proponents?

3. Of those AI academics who believe that a machine intelligence could exist which could outmaneuver humans if motivated, how do they think about the possible motivations of a machine intelligence?
3.1 What fraction of them think about AI design in terms of a formalism such as approximating optimal sequential decision theory under a utility function? How easy would it be for them to substitute anthropomorphic intuitions for correct technical predictions?
3.2 What fraction of them think about AI design in terms of intuitively justified decision heuristics? How easy would it be for them to substitute anthropomorphic intuitions for correct technical predictions?
3.3 What fraction of them understand enough evolutionary psychology and/or cognitive psychology to recognize moral evaluations as algorithmically caused, so that they can reject the default intuitive explanation of the cause of moral evaluations, which seems to be: “there are intrinsic moral qualities attached to objects in the world, and when any intelligent agent apprehends an object with a moral quality, the action of the moral quality on the agent’s intelligence is to cause the agent to experience a moral evaluation”?
3.3.1 What combination of specializations in AI, moral philosophy, and cognitive psychology would an academic need to have, to be an “expert” whose disagreements about the material causes and implementation of moral evaluations were significant?

4. On the question of takeoff speeds, what fraction of the AI academics have a good enough intuitive understanding of decision theory to see that a point estimate or default scenario should not be substituted for a marginal posterior distribution, even in a situation where it would be socially costly in the default scenario to take actions which prevent large losses in one tail of the distribution?
4.1 What fraction recognized that they had a prior belief distribution over possible takeoff speeds at all?
4.2 What fraction understood that, regarding a variable which is underconstrained by evidence, “other people would disapprove of my belief distribution about this variable” is not an indicator for “my belief distribution about this variable puts mass in the wrong places”, except insofar as there is some causal reason to expect that disapproval would be somehow correlated with falsehood?

5 What other popular concerns have academics historically needed to dismiss? What decision rules have they learned to decide whether they need to dismiss a current popular concern?
5.1 After they make a decision to dismiss a popular concern, what kinds of causal explanations of the existence of that concern do they make reference to, when arguing to other people that they should agree with the decision?
5.2 How much do the true decision rules depend on those causal explanations?
5.3 How much do the decision rules depend on intuitions about the concerned peoples’ states of mind and social category memberships?
5.4 How much do the causal explanations use concepts which are implicitly defined by reference to hidden intuitions about states of mind and social category memberships?
5.4.1 Can these intuitively defined concepts carry the full weight of the causal explanations they are used to support, or does their power to cause agreement come from their ability to activate social intuitions?

6. Which people are the AI academics aware of, who have argued that intelligence explosion is a concern? What social categories do they intuit those people to be members of? What arguments are they aware of? What states of mind do they intuit those arguments to be indicators of (e.g. as in intuitively computed separating equilibria)?
6.1 What people and arguments did the AI academics think the other AI academics were thinking of? If only a few of the academics were thinking of people and arguments who they intuited to come from credible social categories and rational states of mind, would they have been able to communicate this to the others?

7. When the AI academics made the decision to dismiss concern about an intelligence explosion, what kinds of causal explanations of the existence of that concern did they intuitively expect that they would be able make reference to, if they later had to argue to other people that they should agree with the decision?

It is also possible to model the social process in the panel:

8. Are there factors that might make a joint statement by a panel of AI academics reflect different conclusions than they would have individually reached if they had been outsiders to the AI profession with the same AI expertise?
8.1 One salient consideration would be that agreeing with popular concern about an intelligence explosion would result in their funding being cut. What effects would this have had?
8.1.1 Would it have affected the order in which they became consciously aware of lines of argument that might make an intelligence explosion seem less or more deserving of concern?
8.1.2 Would it have made them associate concern about an intelligence explosion with unpopularity? In doubtful situations, unpopularity of an argument is one cue for its unjustifiability. Would they associate unpopularity with logical unjustifiability, and then lose willingness to support logically justifiable lines of argument that made an intelligence explosion seem deserving of concern, just as if they had felt those lines of argument to be logically unjustifiable, but without any actual unjustifiability?
8.2 There are social norms to justify taking prestige away from people who push a claim that an argument is justifiable while knowing that other prestigious people think the argument to to be a marker of a non-credible social category or state of mind. How would this have affected the discussion?
8.3 If there were panelists who personally thought the intelligence explosion argument was plausible, and they were in the minority, would the authors of the panel’s report mention it?
8.3.1 Would the authors know about it?
8.3.2 If the authors knew about it, would they feel any justification or need to mention those opinions in the report, given that the other panelists may have imposed on the authors an implicit social obligation to not write a report that would “unfairly” associate them with anything they think will cause them to lose prestige?
8.3.3 If panelists in such a minority knew that the report would not mention their opinions, would they feel any need or justification to object, given the existence of that same implicit social obligation?

9. How good are groups of people at making judgments about arguments that unprecedented things will have grave consequences?
9.1 How common is a reflective, causal understanding of the intuitions people use when judging popular concerns and arguments about unprecedented things, of the sort that would be needed to compute conditional probabilities like “Pr( we would decide that concern is not justified | we made our decision according to intuition X ∧ concern was justified )”?
9.2 How common is the ability to communicate the epistemic implications of that understanding in real-time while a discussion is happening, to keep it from going wrong?

A great breakdown, worth thinking carefully about.

Peter Singer, one of the world’s most influential public intellectuals, and co-author, independent Warsaw-based ethicist Agata Sagan, have published an article called “Rights for Robots?” at the Project Syndicate website. Project Syndicate is “the world’s foremost provider of original commentaries, bringing distinguished voices from around the planet to readers of 432 newspapers in 150 countries.” Other contributors to the site include Bjørn Lomborg, George Soros, Mikhail Gorbachev, and other distinguished persons. Here is the excerpt from the article that mentions SIAI and Eliezer Yudkowsky:

A more ominous question is familiar from novels and movies: Will we have to defend our civilization against intelligent machines of our own creation? Some consider the development of superhuman artificial intelligence inevitable, and expect it to happen no later than 2070. They refer to this moment as “the singularity,” and see it as a world-changing event.

Eliezer Yudkowsky, one of the founders of The Singularity Institute for Artificial Intelligence, believes that singularity will lead to an “intelligence explosion” as super-intelligent machines design even more intelligent machines, with each generation repeating this process. The more cautious Association for the Advancement of Artificial Intelligence has set up a special panel to study what it calls “the potential for loss of human control of computer-based intelligences.”

The panel found that the probability of an intelligence explosion was not great. But see Steve Rayhawk’s analysis for reasons why the chance they would find otherwise would indeed be quite low.

Tonight I will be on FastForward Radio with hosts Phil Bowermaster and Stephen Gordon (and possibly Michael Darling?), to talk about Foresight’s upcoming conference on AGI and nanotech where I will be speaking on anthropomorphism in AGI. The infamous Ralph Merkle will be a guest as well. Tune in at:

10:00 Eastern/9:00 Central/8:00 Mountain/7:00 Pacific.

There will also be a chatroom where you can log in, comment on what we say, and ask questions. Listen live at Blog Talk Radio.

In 2004, Eliezer Yudkowsky of the Singularity Institute presented Coherent Extrapolated Volition (CEV) as a solution to the AI Friendliness Problem. The basic idea is to extrapolate the preferences of all humanity in such a way that we obtain an output that satisfices those preferences, then the CEV shuts down, its role finished. CEV is currently the most promising theory for building a Friendly AI.

A point I haven’t seen advanced before outside this document, though it seems pretty obvious, is that any AI, to be of any use to humans whatsoever, must use some variation of volition to fulfill human directives. Volition is introduced as follows: there are two boxes, box A and box B. One of the boxes has a diamond. Fred wants the diamond, and asks us for box A. We want him to have the diamond. One problem: the diamond is in box B. The document points out the problem with handing Fred box A:

But I do not simply say: “Well, Fred chose box A, and he got box A, so I fail to see why there is a problem.” There are several ways of stating my perceived problem:

1. Fred was disappointed on opening box A, and would have been happier on opening box B.
2. It is possible to predict that if Fred chooses box A, Fred will look back and wish he had chosen box B instead; while if Fred chooses box B, Fred will be satisfied with his choice.
3. Fred wanted “the box containing the diamond”, not “box A”, and chose box A only because he guessed that box A contained the diamond.
4. If Fred had known the correct answer to the question of simple fact, “Which box contains the diamond?”, Fred would have chosen box B.

Hence my intuitive sense that giving Fred box A, as he literally requested, is not actually helping Fred.

If you find a genie bottle that gives you three wishes, it’s probably a good idea to seal the genie bottle in a locked safety box under your bed, unless the genie pays attention to your volition, not just your decision.

A powerful AI, or genie (big difference), must follow our volition, not just our direct decisions, or it would be dangerous. It is easy to imagine even worse failures based on interpreting the letter rather than the spirit of our requests — for instance, a robot chauffeur designed to take one’s children to school would be viewed as an idiotic or evil entity if it took children to school even if the school were on fire, or covered in two feet of snow. Just decisions are never enough — an AI needs an interpretation of volition. I see some connection between the idea of volition and revealed preference — people often say one thing, for social signaling purposes (often subconsciously), when they actually mean something else, which can sometimes be inferred from how they act, not what they say.

To me, the question is not whether we’ll use some form of extrapolated volition to pilot and direct AGI, but what kind we choose to use. In his paper, Eliezer proposed the following:

In poetic terms, our coherent extrapolated volition is our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted.

Phew! That’s a mouthful. What does he mean by “cohere”? What about “growing up farther together”? (I think that should read “further” — “farther” refers to physical distance.) How can we model growing up further together without actually modeling all 6 billion humans interacting socially? Not all these questions are answered in the document. (Some are.) I still regard it as a good starting point. It’s superior to the prior idea that Eliezer had, which was to create an AI that is a “normative altruist” and uses various “anchors and shapers” to craft a “normative morality”. CEV “cheats” by sucking the metamoral content out of the entire human race, like a gigantic infovorous vacuum machine.

The alternative to these sorts of extrapolation schemes all involve a programmer directly dictating the goal content of the AI in one way or another, which leaves you wide open to programmer-biased goal systems. Since the goal system of the first self-improving AI could quite plausibly dictate the fate of the universe from that point on, this is probably a bad idea. Other alternatives, like the one proposed by Bill Hibbard, involve direct feedback where humans essentially push buttons for what they like and the AI is eventually supposed to figure out moral philosophy. (Presumably.) The problem with this is that human metamorality is extremely complex and a system that absorbs the surface features without an eye for deep structure is destined to fail in stupid ways.

Humans can learn more or less what moral behavior is from other humans because much of our metamoral framework is already programmed in from an early age. When a child steals a plate full of cookies that are meant for after dinner, and a parent says, “don’t do that!”, unless the child is extremely young, he or she will generally know what they did wrong and why the adult has a problem with it. A poorly programmed AI, on the other hand, would have no metamoral framework. Was it wrong because cookies are inherently evil? Because the AI did not bake the cookies itself? Because AIs are not meant to have cookies? An AI might know all the facts about cookies and their historical context, but that still won’t give it the background it needs to find out why taking the cookies was “wrong”, and to what extent it was “wrong”. If eating the cookies saved a life, it might not be wrong. What if eating a cookie saved a billion lives a billion years in the the future? A purely utilitarian AI might exterminate the human race today if it thought that doing so would create the greatest utility in the long term. AIs with hand-coded value systems may not have the “moral common sense” that humans do. Moral facts do not follow physical facts. In some cases, we are morally biased for meaningless reasons like small changes in the wording of a hypothetical moral dilemma, or other framing effects. How is an AI supposed to make heads or tails of “right” and “wrong”? Giving up is not a choice — we need an AI we can trust with nuclear weapons or worse. More sophisticated extrapolations of revealed preferences seem to be the most sensible pathway.

The moral realists suppose that a sufficiently intelligent AI will figure out “right” and “wrong” because they are self-evident. This is suicide. Right and wrong are not objective things-in-the-world, but human constructions. Murder is not wrong because it’s objectively wrong, but because human moral development over the course of thousands of years has decided that it is wrong most of the time. People worry about this interpretation of morality because they believe it’s a slippery slope, but Joshua Greene’s PhD thesis goes over all the reasons we might be afraid of moral anti-realism and shows that none of them are really compelling. Whether or not we consider moral anti-realism to be good for society, evolutionary psychology and cognitive science show us that it is true, whether we like it or not.

There is a lot of confusion around the idea of Coherent Extrapolated Volition, which I attribute mostly just to people commenting on the concept without reading the easily digestible 28-page document. People will comment on a concept for years without reading a short document actually explaining it. The way this works is that you read the first page, or less, then fill in the gaps with your imagination.

To dispel some of the worst misconceptions about CEV, here is a short list of “6 points about Coherent Extrapolated Volition” that was posted to the SL4 mailing list in July 2005:

1. Coherent Extrapolated Volition is not a majority vote. No human being is asked to actually decide anything.

2. The key word in “Coherent Extrapolated Volition” is “extrapolated”. CEV does not use judgments produced by the sort of human beings that exist today.

3. The CEV writes an AI. This AI may or may not work in any way remotely resembling a volition-extrapolator.

4. The CEV returns one coherent answer. The AI it returns may or may not display any given sort of coherence in how it treats different people, or create any given sort of coherent world.

5. The CEV runs for five minutes before producing an output. It is not meant to govern for centuries.

6. The CEV by itself does not mess around with your life. The CEV just decides which AI to replace itself with.

For a jumping-off point into one discussion about CEV, see this SL4 thread from Oct. 2008: “Just how coherent does CEV have to be?”, which began with a question proposed by Alex Bokov. Kaj Sotala points out that the initial question is answered in the CEV document itself.

If you have any burning questions, check out the PAQ (Previously Asked Questions) portion of the CEV document first. For another very short summary of the CEV concept, see its Wikipedia entry.

Anders Sandberg directs us to two new lectures by Oxford philosophers.

“Global Catastrophic Risks” by Nick Bostrom
“Human Enhancement: Bioliberation or Biothreat?” by Julian Savulescu

Scroll down a bit to see the controls if you don’t see them at first. The custom flash interface has some cool features, like simultaneously showing the slides and speaker. You can even click a button near the bottom to expand the slides or the speaker window.

In his talk, Savulescu mentions the cognitive enhancement value of iodine in salt. He says that about a billion IQ points are lost each year due to iodine deficiency. If you’re a pregnant woman and you don’t get iodine in your salt during pregnancy, your child loses about 10-15 IQ points. It would cost 2 cents per person per year to iodize salt. 4 billion people lack adequate iodine.

From LifeSiteNews:

WASHINGTON, D.C., November 27, 2009 (LifeSiteNews.com) - President Obama’s brief proclamation of Thanksgiving Day on November 26 was unique among all recorded Thanksgiving proclamations by his predecessors: it is the first one that fails to directly acknowledge the existence of God.

The beneficence shown by God to America is a theme that traditionally defines the Thanksgiving holiday, and this theme is strongly emphasized in the original Thanksgiving Day proclamations and consistently acknowledged even by modern presidents.

Obama’s unprecedented proclamation, however, only makes indirect mention of God by quoting George Washington, stating: “Today, we recall President George Washington, who proclaimed our first national day of public thanksgiving to be observed ‘by acknowledging with grateful hearts the many and signal favors of Almighty God.’”

The proclamation goes on to call Thanksgiving Day “a unique national tradition we all share” that unites people as “thankful for our common blessings.”

“This is a time for us to renew our bonds with one another, and we can fulfill that commitment by serving our communities and our Nation throughout the year,” it continues.

All other presidential Thanksgiving proclamations directly refer to “God,” “Providence,” or another appellation for the divine being.

But Obama’s historic decision to avoid directly mentioning God in the Thanksgiving proclamation doesn’t necessarily come as a surprise. Earlier this year Obama similarly made history on Inaguration Day by explicitly referencing “non-believers” in his speech, which, according to USA Today, was the first time in history that a President had done so. Obama has also said on more than one occasion that the United States is “not a Christian nation.”

Obama’s attitude towards religion is historically unprecedented for an American President.

As somewhat of an aside, Mr. Lynch criticized my critique of Gardner’s theory of “multiple intelligences” as “irreverent”. This is extremely unfair. All I said was that his theory is “something that doesn’t stand up to scientific scrutiny.” I criticize an ad hoc, unscientific theory that has practically no empirical evidence to support it, and the popular appeal of which derives entirely from its egalitarian and inclusive political flavor, and get called irreverent.

Calling Gardner’s theory of multiple intelligences unscientific is not even nearly the most irreverent thing I’ve said, by a long shot. It shouldn’t even be considered irreverent, period. Theories of this sort, which have great popular appeal to the public and practically zero appeal to cognitive psychologists, should be regarded as guilty before proven innocent. Skepticism should be our default mode. Rain on as many unscientific parades as you can.

Dudley Lynch, a self-described “non-scientific observer of what’s being said and written about The Singularity at the moment”, has written up an article on the Singularity. Conclusion: “I suspect it’s still going to be awhile before anyone has an idea about The Singularity worth keeping.”

I get a cameo in his write-up:

Michael Anissimov of the Singularity Institute for Artificial Intelligence and one of the movement’s most articulate voices, continues to warn that “a singleton, a Maximillian, an unrivaled superintelligence, a transcending upload”—you name it—could arrive very quickly and covertly.

Let me add a qualification to that. I do not think that such an entity could arrive quickly and covertly starting from today as a reference point, unless there are extremely well-funded secret projects that have already been working with brilliant researchers and theoreticians for maybe a decade or more (not likely at all). The point I keep making is just that an entity could go quickly from slightly human-surpassing intelligence to superintelligence, a concept known as a “hard takeoff”. To get from here to slightly human-surpassing intelligence could take a while, probably more than 10 years but less than 40 (but who knows), and a project with an annual budget in the millions (maybe tens of millions but probably not hundreds of millions, is my guess). The brain is not magic and we are learning a tremendous amount about it all the time.

I especially stress this point with respect to AI. Even “merely” human-equivalent AI would have a tremendous number of advantages over human thinkers — the ability to copy itself, absorb information more readily, customize and overclock its cognitive modules, design new cognitive modules from scratch, accelerate its thinking speed, avoid the empirically demonstrated biases in reasoning that afflict all humans, explore the entire state space of cognitive features that evolution didn’t think of, blend together deliberative and autonomous cognitive processes, create multiple spheres of attention, and much more. Many of these features are listed in part 3 of “Levels of Organization in General Intelligence”, a Singularity Institute paper.

When us Singularitarians say that an intelligence could potentially bootstrap itself very rapidly from just-barely-smarter-than-human to much-much-smarter-than-human relatively quickly, our reasons aren’t “magic” or “it sounds cool”. We have scientific and rational reasons, it’s just that they don’t fit into soundbites, and there are few people articulate enough to present the arguments in an accessible way.

I don’t personally buy into Kurzweil’s 2029 date — it’s very speculative. The key point is that intelligence operates based on principles and rules that will eventually be reverse-engineered, and once we understand those principles, we’ll have the ability to “teach a rock to think”, to paraphrase Michael Vassar. The ability to teach a rock to think would be no small thing — it could transform the world practically overnight.

Mr. Lynch, here are two ideas about the Singularity worth keeping — one, that artificial intelligences will not behave anthropomorphically, and two, advanced artificial intelligence will be a risk even if we do not program them malignly.

Received from Ben Eisler via Facebook:

Hi everyone,

As some of you may be aware, Peter Thiel (co-founder of PayPal and President of Clarium Capital) is presently committed to matching all donations to the SENS Foundation for aging research by a further fifty percent.

In other words, this means that by reaching our target of ten thousand members, our group has the potential to raise an additional 500 thousand dollars for medical research to end the disabilities and diseases of aging, for everyone.

However, there’s a catch. To take advantage of this considerable matching grant, we must reach our target by the end of this year, giving us just over a month to get there.

We believe we can do it.

We are proposing a massive push so as to make this happen, and we need your help.

If everyone can attract just a few friends to sign up for our cause page, we can reach our target of ten thousand members and raise as much as 1.5 million dollars for SENS research. Ideally we would like to get there by Christmas, as that will give us a week to collect on donations.

This may seem like a tall ask, but remember, there is power in numbers. Once we reach two thousand members, everyone will need attract only four other people to reach our target. Once we reach five thousand, everyone will need to attract only ONE other person, and so on. All very doable!

In the meantime, both SENS Foundation (sens.org) and the Immortality Institute (imminst.org) will be promoting our cause on their websites, which will be a great help as well.

Let’s get to work, and bring an end to the disabilities and diseases of aging.

If 10,000 people all agree to give $100, and Peter Thiel matches it 50%, that equals $1.5M.

Some of us, like those in my generation (I’m 25), may be reluctant to give to SENS because they believe that medical science will progress fast enough without their intervention to let them live indefinitely. I would consider that unfair and calloused to our friends from earlier generations.

Of course, another route to life extension would be through friendly artificial general intelligence (FAI). It’s worth remembering that if we solve the aging problem but not the FAI problem, we all die anyway. However, if we solve the FAI problem but not the aging problem, it’s quite likely that FAI will then solve the aging problem for us. So FAI is truly the only long-term solution for life extension, but SENS is a possible shorter-term solution for the older among us. Even modest success for SENS might also cause the wealthier, older set to start thinking about the longer-term future, which includes the question of how to program powerful artificial intelligences that don’t automatically kill us through indifference.

Audio and video of Richard Feynman’s classic “There’s Plenty of Room at the Bottom” lecture (1959), which presented the vision of molecular nanotechnology for the first time, is available from Photosynthesis.com, an audio site. There are other archival recordings available, including complete audio and video from the 4th Foresight Conference on Molecular Nanotechnology, held in 1995. Apple Computer was a key sponsor of the conference.

Back then, it seems to me that a lot of people thought that molecular nanotech would be closer by now (I remember hearing people say “about 20 years”, so roughly 2015), but they were obviously wrong. My guess is that the innovation and economic activity in the tech sector around that time made them overoptimistic about progress in general.