MASARYK UNIVERSITY FACULTY}w¡¢£¤¥¦§¨  OF I !"#$%&'()+,-./012345

Dangers development of can bring

ESSAY FOR THE COURSE FUTURECHALLENGESOFINFORMATICS

Robert Zahradn´ıˇcek

Brno, Spring 2014 Contents

Introduction ...... 2 Positives, potential and brief history of the development of ..3 Risks and concerns arising from development of the nanotechnology ...... 4 Price we will have to pay ...... 10 Conclusion ...... 11

1 Introduction

Development and progress is important part of our evolution. Our natural passion for creating, enhancing, inventing and discovering is important in many aspects, there is no doubt about that. Trying to stop or limit evolution of our society, in any way, is the same as trying to stop the grass growing on the pavement. You can succeed for while but eventually it will find its way to continue despite your best effort. The same goes for science and technological progress. If we looked back, we would see that such attempts created more evil than good. On the other hand we should never allow the haunt for discoveries to overshadow the basic ethical and rational principles that should guide us in distinguishing good science from bad one. Technological development is driven by the passion of bright people for figuring thinks out and our need for better, faster, simpler and more modern way of life. Modern technologies are therefore becoming more and more encompassed in our day to day life. For this progress we are starting to pay considerable price nowadays. We can no longer imagine life without the digital technologies. We became dependent on them and basically cannot live without them. Also as the development of new technologies progresses, it finds its way to new fields and industries. Furthermore binds us to our own innovations. It is very easy now than ever to forget what is the real cost of these advancements. As any other technology developed by human society, nanotechnologies have numer- ous useful applications, but we shouldn’t be oblivious to the potential dangers such technologies can bring. Therefore we should prepare for what the advanced nanotech- nology can bring and understand its capabilities even before it arrives. So we would be able to use them properly to our benefit. However to make any rational argument, first lets examine both sides, including the background for the debate. There was not even a word ”nanotechnology” in 1959 when Richard Feynman, the Nobel prize laureate, presented his lecture at the California Institute of Technology inspiring the audience, and a lot of people after that, with a view of a working world that was so small that all of the world’s books could be stored on something of the size of a dust speck. In this lecture titled ”There’s Plenty of Room at the Bottom”[3], Richard Feynman said, ”A biological system can be extremely small. Many of the cells are very tiny, but they are active; they manufacture substances; they walk around; they wiggle; and they do all kinds of marvelous things, all on a very small scale. Also, they store information. Consider the possibility that we too can make a thing so small which does what we want, that we can manufacture an object that maneuvers at that level.” Feynman also said, ”I am not inventing anti-gravity, which is possible someday only if laws are not what we think. I am telling you what could be done if the laws are what we think; we are not doing it simply because we haven’t gotten around to it.” Many people will say that Feynman’s speech is the beginning of nanotechnology. However he only stressed that better tools were needed. He spoke of using available tools, the big tools, to make smaller tools suitable for making yet smaller tools, and so on, right to the point where we could manipulate bare atoms.

2 Its doubtful, Feynman envisioned the controversy this lecture and his ideas would produce. Another scientist, , co-founder of Sun Microsystems, published his es- say titled ”Why the future doesn’t need us”[4] in the April 2000 issue of Wired. Bill Joy wrote, ”From the moment I became involved in the creation of new technologies, their ethical dimensions have concerned me, but it was only in the autumn of 1998 that I became anxiously aware of how great are the dangers facing us in the 21st century.” After considerable ratio- nalization, he added, ”Thus we have the possibility not just of weapons of mass destruction but of knowledge-enabled mass destruction, this destructiveness hugely amplified by the power of self replication.”

Positives, potential and brief history of the development of nanotechnology

There is no doubt that nanotechnology has the potential to revolutionize many indus- tries. Many modern problems of our society could be easily fixed by correct deployment of safe advanced nanotechnology. For example being able to analyze thousands microscopic samples of blood simulta- neously by nanosensors, would be great help for diagnostic purposes in medicine. This would essentially automated health diagnosis performed by computer, cutting down the cost, human errors and improving speed of the analysis. We should not need to wait for days for the results, mere minutes would be enough. When we stay in medicine ap- plication, other commonly spoken applications are smart drug delivery for combating diseases, like cancer, or creation of such drugs right at the source. Growing new organs or body parts cell by cell that would perfectly fit the intended recipient, making trans- plant rejection impossible and it could even wipe out the need for transplant donors. If we go even further to the future we could think of automated health centers that would automatically diagnose, detect and repair any known diseases. Practically making state of the art health-care available to everyone right at their homes. Another way to use the potential of the nanotechnology are various enhancements to human bodies, plants, crops or materials, then even immortality would not be so remote. Semiconductor industry is another place where the progress of nanotechnology is visible. Miniaturization of transistors, basic building blocks of modern computer chips, is main goal for creating more powerful, efficient and smaller computing units. Even embedded computing devices have numerous uses for nanotechnology. We could de- velop nanosensors and nanobots and deploy them to the environment around us that could sense, analyze and correct any deviation from normal state. Keeping our living environment clean and safe. Clean and cheap energy provided by solar panels incor- porated with nanotechnology, better water purification, existing pollution reduction, ozone layer repair and many more. These are all good examples of the power a development and correct usage of ad- vanced nanotechnology can provide. All these wonderful applications and examples

3 are what makes the nanotechnology so popular and drives the current research and our understanding to its limits and even beyond. However all of the examples mentioned above have something in common, the in- tended results are based upon assumption that there will be no alternation from the course intended by the creator of that particular technology, there will be no evil pur- poses why such technology was created, no misuse for personal gain, no secret alter- nation of the technology that would be done under the radar etc. That is the theory for imaginary world far from the real one, well at least at this point. Who does not thought at least once in a way, like what will happen if I change this? Or do this? This is the way how the most beautiful and unthinkable things were discovered, by accident or by experiment that revealed unexpected results. But at the same time it gives a space for unintended creations. So this is probably the time and place where we should stop thinking about all the awesome possibilities for a while and become to realize the po- tential dangers such power presents when everything would not be so perfect.

Risks and concerns arising from development of the nanotechnology

Recognition of the two-edged nature of nanotechnology and its ramifications is not something new. It dates back to 1959 and the, mentioned, famous lecture of the physi- cics Richard Feynman ”There is plenty of room at the bottom” [3] or 1986, the publication of ”” [2], a largely positive view of nanotechnology. Or from our century, the article by Bill Joy in the Wired magazine [4]. They all shed some light on what the future of nanotechnology might hold. Although the risks of nanotechnology have been expressed in many fashions over the years, they can be viewed in four distinct ways: nanotechnology puts powerful means of destruction into the hands of irresponsible people; the unknowns of nan- otechnology threaten us with many unintended consequences; nanotechnology can take away our humanity; and nanotechnology contains potential of mass . Much of this is predicated upon the ability of nanodevices to create more nanode- vices, either through the use of specialized assemblers or via self-replication. This could put exquisitely designed weapons into the hands of individuals, terrorists, or rogue nations. Both known weaponry and even new and yet not imagined weapons could become available. With current weapons of mass destruction, testing, obtaining exotic precursors, and specific evidence, make monitoring and control possible in many in- stances. In contrast, weapons created by nanotechnology can be created in such fashion that no current means of monitoring and detection will be applicable. Considering the possibility of nanoweapons, which by definition, are very small and made from com- mon materials, and would be virtually undetectable by any means. Against the background of the many promises of nanotechnology, there is a strong and legitimate concern about the potential dangers of this new capability. The true power of nanotechnology is unproven, but with claims ranging from self-assembling nanodevices to sky hook elevators into space or building entire machines by careful ma-

4 nipulation of bare atoms, the great potential for both good and harm exist within this new technology. The important question is, can we handle all this power? Do we have the social frameworks and necessary knowledge, technical understanding and skills to deal with both the direct and indirect effects of nanotechnology? Lessons from our own history suggests that we probably do not. Aside from the great potential the development of the nanotechnology in medicine holds, there are again growing concerns over its probable toxic effects. Because of the ways it would be possible to use it in medical applications there are concerns about danger to human life. Also other problems are being expressed, like ”Grey Goo problem”, environmental waste from the dead nanoparticals, possible privacy invasion, and more. One can imagine crazy new things can arise, a new age of weapons, specifically designed to avoid the body’s defenses. Similarly, nanodevices, aimed at removing tox- ins and restoring the environment, could be refashioned to destroy crops or selectively cause environmental damage within targeted territory. Unlike , which is indiscriminate and can turn on the aggressor, nanodevices could be programmed to work within boundaries and to self-destruct when their jobs is completed. Leaving no trace of their existence. Monofilaments, if manufacturable, might become essential building blocks for a sky hook and open up space exploration. They could also be used to make nearly invisible and gruesome weapons. Finer than a spider web but as strong as steel, these incredibly thin polymers could slice through almost any materials, making destruction or even murder easier. How we would protect ourselves? When the probably only obstacle that would re- main, is enough understanding and needed skills to design, manufacture and program such devices. Something that is comparable to today information technologies and the relative ease to misuse them by skilled and motivated individuals. Of course, the most celebrated and remarkable promise is the promise of immortal- ity. The flip side of this promise is the risk of extinction itself. We saw enough exam- ples in the past how easy is to destroy billion-dollar project with one simple mistake. Who will ensure that some programmer will not with inadvertently introduced bug in nanorobot’s programming cause massive destruction? Medical applications are very interesting and potentially useful as well, but as it ap- pears, so far, they are quite hard to achieve. Its a lot easier to target healthy cells than navigate and target only the unwanted one. Aside from this, in medicine, nanotechnol- ogy was linked to cases of causing cellular damage, like lungs damage, but it has not been clear how this occurs. However some answers to that were recently given by Ox- ford researchers [6]. This gives us valuable lesson that we should not forget, the power to heal is also the power to harm and can be easily used this way. Probably the most controversial area of research in nanotechnology is the self-replication. While the self-replication would allow easier manufacturing of virtually anything by anybody with considerably lower cost, giving everybody the power to create what they need whenever they need it and want it. Leaving out any need for manufacturing and modern industry as we know it. Thereby making a lot of jobs obsolete, which would

5 destabilize labor market and competitiveness of individuals, organizations and even nations. Because a lot of people would be out of the job and unable to fit to the society of a new age. Reeducation and retraining is not something that can be done over night and it is reasonable to assume that if some significant breakthrough would be made the consecutive impact on everything else would be very rapid, leaving very short time to adapt for this part of the population. Moreover combining such capability with some sort of embedded artificial intelli- gence raises fear of invisible army of devices with their own agenda, ultimately out of our control and for now, even out of our capability to battle something like that. It is reasonable to ask what will happen if we program this devices wrong or the manufacturing process will contain errors? How the precautions should look like? What type of safeguards we should incorporate into the assembler, so we would have the ultimate kill switch in case of something went wrong. In his 1986 book, ”Engines of Creation: The Coming Era of Nanotechnology” [2], K. Eric Drexler said, ”The early transistorized computers soon beat the most advanced vacuum-tube computers because they were based on superior devices. For the same reason, early assembler- based replicators could beat the most advanced modern organisms. ”Plants” with ”leaves” no more efficient than today’s solar cells could out-compete real plants, crowding the biosphere with an inedible foliage. Tough, omnivorous ”bacteria” could out-compete real bacteria: they could spread like blowing pollen, replicate swiftly, and reduce the biosphere to dust in a matter of days. Dangerous replicators could easily be too tough, small, and rapidly spreading to stop—at least if we made no preparation. We have trouble enough controlling viruses and fruit flies”. Among the cognoscenti of nanotechnology, this threat has become known as the ”Gray goo prob- lem”. Though masses of uncontrolled replicating nanodevices need not be particulary gray, the term ”Gray Goo” stresses nanodevices ability to obliterate life might be less in- spiring than a single species of crabgrass. They might be ’superior’ in an evolutionary sense, but this need not make them valuable. He than adds, ”We have evolved to love a world rich in living things, ideas, and diversity, so there is no reason to value gray goo merely because it could spread. Indeed, if we prevent it we will thereby prove our evolutionary superior- ity.” The gray goo threat makes one thing perfectly clear: we cannot afford certain kinds of accidents with replicating assemblers. The gray-goo part especially continues to at- tract a lot of attention from science fiction fans, and some from the media. Really in several sci-fi TV series and movies were pictured nanotechnologies in a way that self- replicating nanobots were created in such a way that they could come together and assemble themselves to any form and shape needed for completing their goal. In some cases there were pictured safeguards that were placed into base of the programming code of the nanobots so they could not harm its creators. That is something that cer- tainly should not be left out from programming code of real nanobots. What happened between Feynman’s speech and Joy’s essay were many discoveries and technological breakthroughs that have led to several Nobel Prizes for their scientific merit and value to humanity, and new fields like with untold promise for breakthroughs in treatments of cancer, diabetes, and other high profile diseases. Clearly

6 there is much support for the view that any potential dangers of nanotechnology to society do not outweigh the potential benefits. Feynman was very positive. He said, ”I am not afraid to consider the final question as to whether, ultimately—in the great future—we can arrange the atoms the way we want; the very atoms, all the way down!” Many people have the ability to create computer viruses, thousands new ones are created each year. In fact, it’s even possible for teenagers with low levels of skill, so- called ”script kiddies” to copy code of somebody else and vandalize the Internet. The combined forces of government and industry have not stopped the creation of new viruses, and there have been numerous instances in our connected world of the quick and destructive spread of these damaging programs. If such capability were extended to the world in general, which is not unlikely event given the pervasiveness of net- works and smart devices, combined with significant availability of assemblers, the re- sults could be tragic. Even if the nanotechnology doesn’t become available to the indi- viduals, it’s likely to be available to many nations. It is not unreasonable to project a significant probability of a costly and risky arms race based on nanotechnology. Advances in nanotechnology are driven almost entirely by curiosity and economics, with little concern for precedents that might be set and the establishment of both moral guidelines and inherent security. This means that it is left to chance who will become the first mover for significant advances, such as the development of assemblers, and how much safety will be built into these devices. There is not even good agreement on when the different advances are likely to take place. In fact, discussions tend to smear out options and possibilities. Speculation about using nanobots to clean the blood stream are likely to be made in the same conversation with predictions of sky hooks and advanced computer components. Going further, there may be even greater challenges if nanotechnology is hybridized in any way with other technologies. No technology then exists in isolation, and the fu- sion of nanotechnology and genetic engineering, or nanotechnology and robotics, could result in even more danger, creating possibility for creating artificial humanoid versions of real people. Probably in early processes for achieving some sort of immortality, where new robotic bodies would be used as shelves for our consciousness. But how much hu- manity would remain in us? There is no way to predict how far would we go from there and if we would even survive as a species. At the same time we had to consider unintended consequences of nanotechnology development. The unknowns of nanotechnology threaten us with pollution and other unintended consequences. Physically, nanodevices have the potential for being persis- tent, like plastics, and invasive, because of their small size. One could easily imagine a worse problem than was seen with DDT in the environment, with last- ing for a thousand years after taking up residence in the tiniest niches of living things. For specific nanomaterials, there may be consequences along the lines of what was dis- covered with ”chemically inert” chlorofluorocarbons, which turned out to catalyze the destruction of the ozone layer. Heat pollution is an unavoidable side effect of the activ- ity of nanodevices. In fact, one defense suggested against a gray goo surprise is simply

7 to look for the heat signature of replicating devices. There’s an old phrase in computing, ”It’s not a bug, it’s a feature.” Bad design raises the risks of unintended consequences. Often designs are done in progressive fashion or without any regard to the actual users and the environment. This sort of engineering approach could have terrible consequences when the realization of the design lies in the real world rather than in cyberspace. Even a good design could easily be extended beyond the creator’s intent. This has been seen in the use of combination drugs for weight loss, with deadly results. In addition, genetic engineering has already witnessed this, when corn not intended for human consumption made its way into the general market, causing allergic reactions among sensitive individuals. The risk of unintended consequences rises steeply in the face of greater capability of nanodevices. If the devices have the ability to replicate, their effect will be similarly amplified. If they are modular, that is, they can be combined or interconnected in some fashion with other nanodevices to form more complex devices, predicting how they might interact with the users and the environment becomes much more difficult. If the devices are purposely evolved rather than designed, a practice that is already in evi- dence in the world of software, then it will become impossible to understand the details of the nanodevices or to predict their effects on the environment. The likelihood and the danger of evolved nanodevices devices will go up significantly if it is proven that evolved software has an economic advantage. Nanotechnology surely can take away our humanity. Inexpensive, invisible, power- ful devices attracts many kinds of people and thereby threaten freedom and privacy if they fall into the wrong hands. A totalitarian regime could use nanodevices to coerce its citizens. Then in the short term, it could monitor them pervasively with tiny sen- sors or threaten them with nanoweapons. It the long-term, modifying the population with nanosurgeries or even using nanobots to transform the genes of future citizens is not something inconceivable. Any sort of holocaust would be possible with nanode- vices to do the work, large sectors of the population could be selectively destroyed, and intervention by other nations could be discouraged by threats based on expertise in nan- otechnology. Basic human right would be absolutely useless and no longer enforceable in any reasonable way. A more subtle threat is that represented by the cyborg. Embedding nanodevices into our bodies as part of medical procedures or either to extend our powers or to extend our lives, creates an intimate relationship between our nanomachines and ourselves. Whether the result is viewed as symbiotic or parasitic, at some point, the needs, values, and orientation of these new individuals may become significantly different from what is currently defined as human. To some, this is an opportunity to revel in, but there is by no means a consensus as to whether this is good for our kind or not. The social conse- quences could be profound. Are unmodified humans obsolete or possibly expendable? Is the question something that we should be asking? As people become cyborgs, are they still part of our community with all the same rights? How would the accumulation of power and wealth by ”immortals” be handled? How are benefits distributed? Who is responsible for the costs of side effects? Philosophically, if all natural parts are replaced

8 by nanodevices, and the resulting individual passes the Turing test, do you still have a human? Is it a good thing for the species if all humans are replaced this way, or is it an empty fantasy that ends humanity? Questions that hardly anybody can begin to give answers to. Is it necessary to fight an evolution in so unnatural way? At the same time as the nanotechnology can make us immortal it contains the seeds of mass extinction. In 1974, leading scientists in the field of genetic engineering called for a moratorium on certain areas of research. In 1975, the Asilomar agreement went into effect, and scientists voluntarily suspended their work until the consequences were explored, rules were written, and remedies were found. But no such agreement has been made among leading nanotechnologists so far. Is the threat of nanotechnology less obvious or less real than what the genetic engi- neers faced? This is really unlikely. Genetic engineering had its Andromeda Strain con- cerns just as the gray goo haunts nanotechnology and both had and have their healthy skeptics. In 1975, genetic engineering was still in its infancy, with cloning, embryo re- search, ”Frankenfood,” and gene therapy still many years in the future. The primitive state of the art of nanotechnology doesn’t explain the lack of attention devoted to its po- tential dangers. There are, however, two key differences between nanotechnology and earlier technological advances: the experiences of the scientists and the funding of their research. Asilomar was an offshoot of the Pugwash conferences, where scientists took on the tough issues of living with nuclear weapons. The threat of nuclear annihilation was more vivid in the polarized Cold War years. Testing, proliferation, and safety problems were both urgent and intractable. Scientists were at the center of nuclear debates and were asked to make policy recommendations. Many also felt guilty about developing the Bomb and not opposing its use in Japan. Although the issues of living with nuclear weapons haven’t gone away, they have faded—and almost no nanotechnologists have personal experience with building weapons of mass destruction. While funding for genetic engineering was largely in the hands of government, specifically the National Institutes of Health (NIH), nanotechnology gets much of its funding from industry. Its pressures are market pressures, and competition encourages speed and applicability over deliberation and understanding. A nanotechnologist who advocates a pause in the development of nanotechnology would not only be challeng- ing the current market ethos but taking on the corporation that funds his or her work. Nanotechnologists are not restrained by their experiences and are driven by their funders to put nanodevices into the real world. In this context, it seems unlikely that precautions like secrecy, testing, and isolation of nanodevices will be taken. It is prob- able that nanodevices will not be engineered for safety with features such as traceabil- ity, self-destruction, and dependency. All this makes abuse more likely and error more probable. The consequences can range from vandalism to the infamous gray goo. The power of nanotechnology is not inherently bad, but its destructive potential is great. The legal, technical, and cultural work that must be done to tip the balance toward safety is in my opinion not being conducted. Laws are needed to control in- formation, proscribe areas of activity, and control use and distribution. We must, as a

9 society, develop a cultural response to this power that includes education, discussion, and approaches toward agreement and consensus. Timing is everything. Unlike our ex- perience with nuclear weapons, we still have the time to be thoughtful, to prepare. But today, with work on safety still in front of us, we face a dangerous future. If Bill Joy is correct, that future is not far away.

Price we will have to pay

With the proliferation of nanotechnology research, the driving forces of a worldwide economy that is tied to the advance of electronics, and the society that demands medical progress, the question of whether the potential dangers of nanotechnology to the society outweigh the potential benefits may be an entirely academic debate. The genie is out. Society has tasted the successes of nanotechnology and they are many. Even with the science fictionese suggestion of replicating robots, gray goo and all, nanotechnology is here to stay. The dangers, problems, concerns are out there, spoken loudly and clearly many times over and over. While the progress is inevitable we should be prepared to what it will cost us. Of the hundreds of nano-enhanced products already on the market, many are cos- metics, and many others, such as clothing and computer peripherals, are spiked with silver for unnecessary antibacterial effects. Convenience items are another big category. Who needs this stuff? Easy question and some environmental groups are calling for moratorium on nano-containing products. It appears that nanotechnology is slowly being embedded to everything around us without us even noticing. In retrospective its terrifying idea. We are unwillingly loosing our free will to choose. Becoming dependent upon products of daily use. We do not have to go far to imagine how easy it would be to even develop some sort of addic- tion. Is it worth it? Another question that is not easy to answer. The answer probably depends on the individual view of the nanotechnology. If one considers the viewpoint, that nanotechnology research contributions far outweigh any perceived dangers, then the answer should be, yes it is worth it. How reasonable is this point of view is ques- tionable at best. Maybe the price paid will be worth it. Even today people are willing to sacrifice their privacy and control over their lives for little bit of comfort. Or maybe not. If we agree that nanotechnology holds significant risks and that work that must be done to tip the balance toward safety is not being done. Then it is not worth it. This is however mere speculation. I cannot foresee how the future will play out or predict how the future society will react to these ongoing changes. Looking to the past I see hope that someone will always spread reason so we will not have to pay the ulti- mate price with our lives. Yet sometimes a lie said thousand times can become the truth. Those with open eyes can see it everyday, how we are being fed with half-truths by me- dia. This two sides of a coin, two-edged nature of nanotechnology, make predictions difficult and interesting at the same time. I would say that the amount of trustworthy people around us that will pioneer usage of nanotechnology will be the important factor

10 in the future and the price we will ultimately have to pay for our advances in nanotech- nology.

Conclusion

At this point many of the most basic questions about nanohazards are unfortunately unanswered. What materials are harmful, in what particle sizes and shapes, under what conditions? Who is at risk: Workers? People using nano-enabled products? Wildlife and ecosystems? How should we measure exposures? It is important to find scientific an- swers for these questions and work out safe procedures for manufacturing and R&D. Therefore protecting workers and researches from unintentional harm caused by im- proper methods and protocols. At the same time there should be some consensus about from which materials it is safe to manufacture the nanobots and make laws and gov- ernment organs to monitor and enforce proper processes. Finally, probably the most problematic issue will be establishing social frameworks that would enable reasonable trust to manufacturers and government that they would not misuse these technology for unethical purposes and private agendas of powerful individuals thereby enlarging the gap between rich and powerful part of our society and the rest. I see the only way to get through this problem will be proper education of population so they would not blindly trusting and using unsafe products the way it is happening today with many information technologies thereby providing ground for cyber-attacks and frauds of dif- ferent kind. Other part should be our own ethical and moral principles that would make misuse of any kind unthinkable. But that is probably even more far away than anything else. While I strongly believe that the progress is necessary and inevitable, we should not allow these advancements to have power over common ethical and rational principles. As any other technology, nanotechnology should be governed by reason and safety poli- cies and not rushed to production because we need them or because people behind the funding want their investments back as soon as possible. After all, sometimes is bet- ter to wait for a while and have good and safe product than rush to the production and end up with faulty one, which in many applications of advanced nanotechnology would cost human lives. On the other hand limiting the progress just because we do not understand it well enough or even banning it altogether is not the proper way how to address arise of nanotechnology. ” [1] Leslie White wrote in his book[5] ”Social systems are determined by technological sys- tems”. But I would like to argue here a little bit. I agree that this holds true when we look on the society through the glasses of technological advancements, but does not the way we use what we created matter too? We created many powerful things that emerged from pure ideas but our history shows us that while we can think of beautiful new things we are quite far from being equally capable of using them appropriately. I mentioned many times on the lines above how important is to be guided by correct rational a moral principle and I would like to stress it once more.

11 Bibliography

[1] Center for Responsible Nanotechnology - Position. http://www.crnano.org/positions.htm. Accessed 29.6.2014.

[2] K. Eric Drexler. Engines of Creation: The Coming Era of Nanotechnology. Anchor Books, 1986.

[3] Richard Feynmann. There’s Plenty of Room at the Bottom. http://en.wikipedia.org/wiki/There’s Plenty of Room at the Bottom, 1959. Accessed 3.6.2014.

[4] Bill Joy. Why the future doesn’t need us. http://archive.wired.com/wired/archive/8.04/joy pr.html, 2004. Accessed 29.6.2014.

[5] Leslie A. White, . The Evolution of Culture: The Development of Civilization to the Fall of Rome. Left Coast Press, 1959.

[6] Chenggang Li, Haolin Liu, Yang Sun, Hongliang Wang, Feng Guo, Shuan Rao, Jiejie Deng, Yanli Zhang, Yufa Miao, Chenying Guo, et al. PAMAM promote acute lung injury by inducing autophagic cell death through the Akt-TSC2-mTOR signaling pathway. Journal of molecular cell biology, 1(1):37–45, 2009.

12