Google to Host Terabytes of Open-Source Science Data

• By Alexis Madrigal
• January 18, 2008  | 

Sources at Google have disclosed that the humble domain, http://research.google.com, will soon provide a home for terabytes of open-source scientific datasets. The storage will be free to scientists and access to the data will be free for all. The project, known as Palimpsest and first previewed to the scientific community at the Science Foo camp at the Googleplex last August, missed its original launch date this week, but will debut soon.
Building on the company’s acquisition of the data visualization technology, Trendalyzer, from the oft-lauded, TED presenting Gapminder team, Google will also be offering algorithms for the examination and probing of the information. The new site will have YouTube-style annotating and commenting features.
The storage would fill a major need for scientists who want to openly share their data, and would allow citizen scientists access to an unprecedented amount of data to explore. For example, two planned datasets are all 120 terabytes of Hubble Space Telescope data and the images from the Archimedes Palimpsest, the 10th century manuscript that inspired the Google dataset storage project.
UPDATE (12:01pm): Attila Csordas of Pimm has a lot more details on the project, including a set of slides that Jon Trowbridge of Google gave at a presentation in Paris last year. WIRED’s own Thomas Goetz also mentioned the project in his fantastic piece of freeing dark data.

One major issue with science’s huge datasets is how to get them to
Google. In this post by a SciFoo attendee over at business|bytes|genes|molecules, the collection plan was described:

(Google people) are providing a 3TB drive array (Linux RAID5). The array is provided in “suitcase” and shipped to anyone who wants to send they data to Google. Anyone interested gives
Google the file tree, and they SLURP the data off the drive. I believe they can extend this to a larger array (my memory says 20TB).

You can check out more details on why hard drives are the preferred distribution method at Pimm. And we hear that Google is hunting for cool datasets, so if you have one, it might pay to get in touch with them.

Freeing the Dark Data of Failed Scientific Experiments

By Thomas Goetz 09.25.07

Photo: Mauricio Alejo 

 

In 1981, the New England Journal of Medicine published a Harvard study that showed an unexpected link between drinking coffee and pancreatic cancer. As it happened, researchers were anticipating a connection between alcohol or tobacco and cancer. But according to the survey of several hundred patients, booze and cigarettes didn't seem to increase your risk. Then came a surprise: An incidental survey question suggested that coffee did increase the chances of pancreatic cancer. So that's what got published.

Those positive results, alas, were entirely anomalous; 20 years of follow-up research showed the coffee-cancer connection to be bunk. Nonetheless, it's a textbook example of so-called publication bias, where science gets skewed because only positive correlations see the light of day. After all, the surprising findings are what makes the news (and careers).

So what happens to all the research that doesn't yield a dramatic outcome — or, worse, the opposite of what researchers had hoped? It ends up stuffed in some lab drawer. The result is a vast body of squandered knowledge that represents a waste of resources and a drag on scientific progress. This information — call it dark data — must be set free.

For the past couple of years, there's been much talk about open access, the idea that more scientific publications should be freely available — not locked behind firewalls and subscriptions. Thanks to the Public Library of Science (PLoS) and other organizations, that notion is making headway. Liberating dark data takes this ethos one step further. It also makes many scientists deeply uncomfortable, because it calls for them to reveal their "failures." But in this data-intensive age, those apparent dead ends could be more important than the breakthroughs. After all, some of today's most compelling research efforts aren't one-off studies that eke out statistically significant results, they're meta-studies — studies of studies — that crunch data from dozens of sources, producing results that are much more likely to be true. What's more, your dead end may be another scientist's missing link, the elusive chunk of data they needed. Freeing up dark data could represent one of the biggest boons to research in decades, fueling advances in genetics, neuroscience, and biotech.

So why doesn't it happen? In part, it's a logistics problem: Advocating the release of dark data is one thing, but it's quite another to actually collect it, juggling different formats and standards. And, of course, there's the issue of storage. These days, an astronomical study of quasars or an ambitious bioinformatics project can generate several terabytes of data. Few have the capacity to store that, let alone analyze it.

Google, among others, is lending a hand with its Palimpsest project, offering to store and share monster-size data sets (making the data searchable isn't a part of the effort). As storage costs drop, similar data banks will emerge, along with format standards, and it should become ever easier to share results, good or bad.

Technology is actually the simple part. The tougher problem lies in the culture of science. More and more, research is funded by commercial entities, which deem any results proprietary. And even among fair-minded academics, the pressures of time, tender, and tenure can make openness an afterthought. If their research is successful, many academics guard their data like Gollum, wringing all the publication opportunities they can out of it over years. If the research doesn't pan out, there's a strong incentive to move on, ASAP, and a disincentive to linger in eddies that may not advance one's job prospects.
There are some islands of innovation. Since 2002, the Journal of Negative Results in Biomedicine has offered a peer-reviewed home to results that go negative or against the grain. Earlier this year, the journal Nature started Nature Precedings, a Web-based forum for prepublication research and unpublished manuscripts in biomedicine, chemistry, and the earth sciences. At Drexel University, chemist Jean-Claude Bradley practices "open notebook" science — chronicling his lab's work and sharing data via blog and wiki. And PLoS is planning an open repository for research and data that is other wise abandoned.

These are great first steps. But freeing dark data should be the norm, not the exception. Once the storage and format problems are solved, scientists will need easy ways to search and retrieve each other's data. (And there should be a simple way, perhaps via metadata, to make sure that work is always duly cited and acknowledged by others.) Congress should mandate that all federally funded research be disseminated, whatever the results.

Getting science comfortable with exposing its dark data is really just the beginning. Once you start looking for it, dark data is everywhere: It's locked away in out-of-print books and orphaned art, the stuff that Creative Commons and Google Book Search have been bringing to light. Speaking of which: Hey, Google! Know all those research projects your employees do that the company will never green-light? How about letting the rest of the world take a crack at them?

Deputy editor Thomas Goetz (thomas@wired.com) wrote about DNA diagnostics in issue 15.08.

The Thirteen Mindfulness Trainings

The Thirteen Mindfulness Trainings form the moral guidelines to develop harmony in any simple community. One of the essential elements of the Mindfulness Trainings is that they are directly applied in our daily lives. Every moment of our lives gives us the chance to put them into practice.

The idea is to recite the Mindfulness Trainings regularly so that we can review our behaviour and observe where we have not lived up to our aspirations. It is important that we observe rather than judge ourselves, so that gradually, without resistance, our lives become imbued with the qualities they represent.

Understanding (prajna), concentration (samadhi) and Mindfulness Trainings or ethics (sila) are the threefold trainings that the Buddha passed on to his lay students. The practice of each of these trainings is equally important.

The encounter between Eastern philosophies and the West is bringing us something very exciting, very important. When Buddhism enters one country, that country always acquires a new form of Buddhism. The merge result must be suitable, appropriate to the psychology and the cul ture of the society that it serves.

The Thirteen Mindfulness Trainings comes from different sources. The last part of them (5) comes directly from the Five Precepts followed by all the Tibetan Buddhist schools (Sila). The first eight comes from the Tiep Hien Order founded in Vietnam during the war and which came from the Zen School of Lin Chi.

Tiep and Hien are Vietnamese words of Chinese origin. I would like to explain the meaning of these words, because understanding them helps in understanding the spirit of the Trainings.

Tiep means “to be in touch.” First of all, to be in touch with oneself in order to find out the source of wisdom, understanding, and compassion in each of us. Being in touch with oneself is the meaning of medi tation, to be aware of what is going on in your body, in your feelings, in your mind. That is the first mean ing of Tiep.

Tiep also means to be in touch with Buddhas and Bodhisattvas, the enlightened people in whom full understanding and compassion are tangible and ef fective. Being in touch with oneself means being in touch with this source of wisdom and compassion.

Hien means “the present time.” We have to be in the present time, because only the present is real, only in the present moment can we be alive. We do not practice for the sake of the future, to be reborn in a paradise, but to be peace, to be compassion, to be joy right now.

The First Mindfulness Training: Openness

Aware of the suffering created by fanaticism and intolerance, I am determined not to be idolatrous about or bound to any doctrine, theory or ideology, even Buddhist ones. Buddhist teachings are guiding means to help me learn to look deeply and to develop my understanding and compassion. They are not doctrines to fight, kill or die for.

The Second Mindfulness Training: Non-attachment to Views

Aware of the suffering created by attachment to views and wrong perceptions, I am determined to avoid being narrow-minded and bound to present views. I will learn and practise non-attachment from views in order to be open to others’ insights and experiences. I am aware that the knowledge I presently possess is always changing and not an absolute truth. Truth is found in life and I will observe life within and around me in every moment, ready to learn throughout my life.

The Third Mindfulness Training: Freedom of Thought

Aware of the suffering brought about when I impose my views on others, I am committed not to force others, even my children, by any means whatsoever, to adopt my views. I will respect the right of others to be different and to choose what to believe and how to decide.

The Fourth Mindfulness Training: Awareness of Suffering

Aware that looking deeply at the nature of suffering can help me develop compassion and find ways out of suffering, I am determined not to avoid or close my eyes before suffering. I am determined to be with those who suffer, so I can understand their situation deeply and help them transform their suffering into compassion, peace and joy.

The Fifth Mindfulness Training: Living Simply

Aware that true happiness is rooted in peace, solidarity, freedom and compassion, and not in wealth or fame, I am determined not to take as the aim of my life fame, profit, wealth, or sensual pleasure. I am committed to living simply and sharing my time, energy and material resources with those around me in need, including my blood and spiritual family, my friends, my ancestors and my descendents.

The Sixth Mindfulness Training: Dealing with Anger

Aware that anger blocks communication and creates suffering, I am determined to take care of the energy of anger when it arises and to recognize and transform the seeds of anger that lie deep in my consciousness. When anger comes up, I am determined not to do or say anything, but to find the best way to ease my anger. I will learn to look with the eyes of compassion on those I think are the cause of my anger.

The Seventh Mindfulness Training: Dwelling Happily in the Present Moment

Aware that life is available only in the present moment and that it is possible to live happily in the here and now, I am committed to training myself to live deeply each moment of daily life. I will try not to lose myself in dispersion or be carried away by regrets about the past, worries about the future, or cravings, anger or jealousy in the present. I will practise mindful breathing to come back to what is happening in the present moment. I am determined to learn the art of mindful living by touching the wondrous, refreshing and healing elements that are inside and around me, and by nourishing seeds of joy, peace, love and understanding in myself, thus facilitating the work of transformation and healing in my consciousness.

The Eighth Mindfulness Training: Communication and Reconciliation

Aware that lack of communication always brings separation and suffering, I will make every effort to keep communications open and to reconcile and resolve all conflicts, however small. Aware of the suffering caused by unmindful speech and the inability to listen to others, I am committed to cultivating loving speech and deep listening in order to bring joy and happiness to others and relieve others from their suffering.

The Ninth Mindfulness Training: Truthful Speech

Aware that words can create suffering or happiness, I am committed speak truthfully and constructively, using only words that inspire hope and confidence. I am determined not to say untruthful things for the sake of personal interest or to impress people, nor to utter words that might cause division or hatred. I will not spread news that I do not know to be certain nor criticize or condemn things of which I am not sure.

It is important to remember the four faults by the word:

• lying.
• pronouncing offensive words (hurtful speech).
• uttering words that can cause division between two or more people by talking negatively of some of them, spread rumors, or sowing doubts. We have to be careful not to increase discord between people but to make all efforts to reconcile and resolve all conflicts, however small.
• indulging in idle chatter. We must abstain of any kind of unmindful speech; spreading news we do not know to be certain; criticize or condemn people.

Our speech must embody mindfulness and we must be completely aware and responsible while pronouncing words.

Try making your words wiser than the silence that has been broken.

The Tenth Mindfulness Training: Reverence for Life

I am committed to abstain from doing harm to any sensitive beings. This commitment implies the determination to respect the life of all beings, not only human beings but also all animals and plants, regardless of their size; and an increasing awareness of the sanctity of life. Moreover it means more than a respect for the sacredness of life since it means a commitment to neither harm nor offend anyone under any circumstances and to promote peace and peace education.

The Eleventh Mindfulness Training: Respect and Generosity

I am committed to abstaining from taking what has not been given to me. This commitment implies the respect for other beings’ property. It not only implies not to steal, but encourages us to cultivate the attitude of not to demand or claim what we wish, but to learn ways to work for the benefit of people, animals and plants, practicing generosity, sharing our time, energy and resources with those in need, and to patiently wait to receive the results of our good actions.

The Twelfth Mindfulness Training: Correct use of our Senses

I am committed to abstain from the wrong use of my senses. This commitment implies contemplating our body as a temple that we take care of and cultivate for spiritual growth. What is advised here is to abstain from the disorderly use of the body, sexuality and body sensations, avoiding falling into gluttony, lust, incest and all kinds of disorders produced by inadequate sensory behavior.

The Thirteenth Mindfulness Training: Mindful Consumption

Aware of the suffering caused by unmindful consumption, I am committed to abstain from consuming items that cloud or blur the mind. It not only implies to abstain from alcohol and intoxicating drugs but to take care of the quality of information that we are consuming through all our senses, what are we reading, what we are seeing, who we are joining. We are aware that everything we consume will feed our mind and the collective mind of our family and society.