Go Back   nV News Forums > General Forums > Archived News Items

Newegg Daily Deals

Thread Tools
Old 04-12-12, 03:00 PM   #1
Registered User
Join Date: Jun 2009
Posts: 56,596
Post For Performance, Keep File Counts Small

Brock Palen writes that the act of writing too many small files can kill application performance, but there is a remedy.
Lustre meta-data (the existence of the file) lives on its' own server. There is only one for an entire file system. That huge 10?s PB filesystem? Only 1 meta-data server. This can be a bottle neck. To open a file, first the client talks to this MDS (meta-data server) which tells the client which OSS (storage server) to write data to. Lustre will have many OSS's. If the client keeps creating new files or opening and closing the same file, it keeps making that trip back to that single MDS. If the client creates one file, doesn't close it, and keeps writing to it, the client never speaks to the MDS again! Just to the, many, OSS nodes. Obviously the client can avoid making the extra network trip over to the MDS and back multiple times, but it also avoids this single server bottle neck.

Read the Full Story.

Related posts:

News is offline   Reply With Quote

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

All times are GMT -5. The time now is 03:58 PM.

Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2015, Jelsoft Enterprises Ltd.
Copyright 1998 - 2014, nV News.