Go Back   nV News Forums > General Forums > Archived News Items

Newegg Daily Deals

Thread Tools
Old 06-27-12, 08:30 AM   #1
Registered User
Join Date: Jun 2009
Posts: 56,600
Post Sifting Through a Trillion Electrons

What happens when your scientific instrument generates more data than you can decipher? Linda Vu writes that Berkeley researchers are designing new strategies for extracting interesting data from massive scientific datasets.

In a recent case, a team ran a state-of-the-art plasma physics code called VPIC on NERSC's Cray XE6 'Hopper' supercomputer that generated a 3D magnetic reconnection dataset of a trillion particles.
Although our VPIC runs typically generate two types of data'grid and particle'we never did a whole lot with the particle data because it was really hard to extract information from a trillion particle dataset, and there was no way to sift out the useful information,' said Homa Karimabadi, head of the space physics group at UCSD.

Using their new tools, the researchers wrote each 32 TB file to disk in about 20 minutes, at a sustained rate of 27 gigabytes per second. By applying an enhanced version of the FastQuery tool, the team indexed this massive dataset in about 10 minutes, then queried the dataset in three seconds for interesting features to visualize.

Read the Full Story.

Related posts:

News is offline   Reply With Quote
Old 06-27-12, 02:02 PM   #2
Registered User
Join Date: Aug 2002
Location: San Jose, CA
Posts: 449
Default Re: Sifting Through a Trillion Electrons

Yup, presenting huge amount of data in a meaningful way is a *very* difficult problem.
As the universe is curved, there cannot be a straight answer...
frenchy2k1 is offline   Reply With Quote

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
nvidia: 34 trillion calculations to evaluate 1 sq km area, and ID if appropriate for News Archived News Items 0 05-17-12 10:40 PM

All times are GMT -5. The time now is 10:12 PM.

Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2015, Jelsoft Enterprises Ltd.
Copyright 1998 - 2014, nV News.