MetaDb Class Reference

#include <metadb.h>

Detailed Description

MetaDb class is container for all Meta Data handled by this application.

It provides a number of lookup functions as well as addition functions publically. Internally, it stores various cross-referenced lists to provide fast lookups. The contents of this class are stored in metadb.xml, and loaded on each runtime.

The purpose of this class is to provide a application-central database through which it is possible to perform cross-referencing between FilesList and this list.

Definition at line 45 of file metadb.h.


Public Member Functions
void	load (std::istream &is)
	Load the MetaDb contents from input stream, adding all found entries in the stream to the database, merging duplicate entries.
void	save (std::ostream &os) const
	Write the contents of the database into output stream.
void	push (MetaData *md)
	Add a metadata object to the database.
void	push (MetaData md, SharedFile sf)
	Add a metadata object to the database, associating it with a SharedFile.
MetaData *	find (const HashBase &h) const
	Find Metadata by searching with hash.
std::vector< MetaData * >	find (const std::string &filename) const
	Find MetaData by searching with file name.
MetaData *	find (SharedFile *sf) const
	Find MetaData by searching with SharedFile.
SharedFile *	findSharedFile (const HashBase &h) const
	Locate SharedFile by searching with hash.
std::vector< SharedFile * >	findSharedFile (const std::string &filename) const
	Locate SharedFile by searching with file name.
Static Public Member Functions
static MetaDb &	instance ()
	This class is a Singleton.
Private Types
typedef std::set< MetaData * >::iterator	LIter
	List Iterator.
typedef std::set< MetaData * >::const_iterator	CLIter
	Constant List Iterator.
typedef std::map< SharedFile , MetaData >::iterator	SFMDIter
	SharedFile to MetaData Iterator.
typedef std::map< SharedFile , MetaData >::const_iterator	CSFMDIter
	Constant SharedFile to MetaData Iterator.
typedef std::multimap< std::string, MetaData * >::iterator	FNIter
	File Name Iterator.
typedef std::multimap< std::string, MetaData * >::const_iterator	CFNIter
	Constant File Name Iterator.
typedef std::multimap< std::string, SharedFile * >::iterator	NTSFIter
	Name To SharedFile Iterator.
typedef std::multimap< std::string, SharedFile * >::const_iterator	CNTSFIter
	Constant Name To SharedFile Iterator.
typedef std::map< CGComm::HashTypeId, std::map< HashWrapper, SharedFile * > >::iterator	HTSFIter
	Hash To SharedFile Iterator.
typedef std::map< CGComm::HashTypeId, std::map< HashWrapper, SharedFile * > >::const_iterator	CHTSFIter
	Constant Hash To SharedFile Iterator.
typedef std::map< HashWrapper, SharedFile * >::iterator	HWTSFIter
	HashWrapper To SharedFile Iterator.
typedef std::map< HashWrapper, SharedFile * >::const_iterator	CHWTSFIter
	Constant HashWrapper To SharedFile Iterator.
typedef std::map< CGComm::HashTypeId, std::map< HashWrapper, MetaData * > >::iterator	HTMDIter
	Hash To Meta Data Iterator.
typedef std::map< CGComm::HashTypeId, std::map< HashWrapper, MetaData * > >::const_iterator	CHTMDIter
	Constant Hash To Meta Data Iterator.
typedef std::map< HashWrapper, MetaData * >::iterator	HWTMDIter
	Hash Wrapper To Meta Data Iterator.
typedef std::map< HashWrapper, MetaData * >::const_iterator	CHWTMDIter
	Constant Hash Wrapper To Meta Data Iterator.
Private Member Functions
void	onMetaDataEvent (MetaData *md, int evt)
	Event handler for MetaData events, called from event table.
void	onSharedFileEvent (SharedFile *sf, int evt)
	Event handler for SharedFile events, called from event table.
void	tryAddFileName (MetaData *source, const std::string &name)
	Attempt to add entry to m_filenames list.
void	tryAddFileName (SharedFile *sf, const std::string &name)
	Attempt to add entry to m_nameToId map.
void	tryAddHashSet (MetaData source, const HashSetBase hash)
	Attempt to add entry to m_hashes list.
void	tryAddHashSet (SharedFile sf, const HashSetBase hash)
	Attempt to add entry to m_hashToId map.
void	clear ()
	Clears all lists.
Singleton
	MetaDb ()
	MetaDb (MetaDb &)
MetaDb &	operator= (const MetaDb &)
	~MetaDb ()
Private Attributes
std::set< MetaData * >	m_list
	Primary List ------------ Pure metadata objects.
std::map< SharedFile , MetaData >	m_sfToMd
	SharedFile To MetaData Map ---------------------- SharedFile* keyed map to locate MetaData objects by searching with SharedFile pointer.
std::multimap< std::string, MetaData * >	m_filenames
	File Name to MetaData Map ------------------------- This map allows finding MetaData knowing file name.
std::multimap< std::string, SharedFile * >	m_nameToSF
	File Name to SharedFile Map ------------------------ This map allows searching with file name and locating the corresponding SharedFile's.
std::map< CGComm::HashTypeId, std::map< HashWrapper, SharedFile * > >	m_hashToSF
	Hash To SharedFile Map ---------------------- A dual map which allows us to locate SharedFiles given a hash.
std::map< CGComm::HashTypeId, std::map< HashWrapper, MetaData * > >	m_hashes
	Hash To Meta Data Map --------------------- Last, but not least, a recursive map which allows us to look up MetaData objects given we know one hash.
Friends
std::ostream &	operator<< (std::ostream &o, const MetaDb &md)
	Output operator for streams.
void	test_metadb ()
Classes
class	HashWrapper
	HashWrapper structure acts as a container for HashBase pointers in order to allow us to have a map of those keyed by the Hash. More...

Member Typedef Documentation

typedef std::multimap<std::string, MetaData*>::const_iterator MetaDb::CFNIter [private]

Constant File Name Iterator.

Definition at line 217 of file metadb.h.

typedef std::map< CGComm::HashTypeId, std::map<HashWrapper, MetaData*> >::const_iterator MetaDb::CHTMDIter [private]

Constant Hash To Meta Data Iterator.

Definition at line 286 of file metadb.h.

typedef std::map< CGComm::HashTypeId, std::map<HashWrapper, SharedFile*> >::const_iterator MetaDb::CHTSFIter [private]

Constant Hash To SharedFile Iterator.

Definition at line 253 of file metadb.h.

typedef std::map<HashWrapper, MetaData*>::const_iterator MetaDb::CHWTMDIter [private]

Constant Hash Wrapper To Meta Data Iterator.

Definition at line 290 of file metadb.h.

typedef std::map<HashWrapper, SharedFile*>::const_iterator MetaDb::CHWTSFIter [private]

Constant HashWrapper To SharedFile Iterator.

Definition at line 257 of file metadb.h.

typedef std::set<MetaData*>::const_iterator MetaDb::CLIter [private]

Constant List Iterator.

Definition at line 188 of file metadb.h.

typedef std::multimap< std::string, SharedFile*>::const_iterator MetaDb::CNTSFIter [private]

Constant Name To SharedFile Iterator.

Definition at line 232 of file metadb.h.

typedef std::map<SharedFile*, MetaData*>::const_iterator MetaDb::CSFMDIter [private]

Constant SharedFile to MetaData Iterator.

Definition at line 203 of file metadb.h.

typedef std::multimap<std::string, MetaData*>::iterator MetaDb::FNIter [private]

File Name Iterator.

Definition at line 215 of file metadb.h.

typedef std::map< CGComm::HashTypeId, std::map<HashWrapper, MetaData*> >::iterator MetaDb::HTMDIter [private]

Hash To Meta Data Iterator.

Definition at line 282 of file metadb.h.

typedef std::map< CGComm::HashTypeId, std::map<HashWrapper, SharedFile*> >::iterator MetaDb::HTSFIter [private]

Hash To SharedFile Iterator.

Definition at line 249 of file metadb.h.

typedef std::map<HashWrapper, MetaData*>::iterator MetaDb::HWTMDIter [private]

Hash Wrapper To Meta Data Iterator.

Definition at line 288 of file metadb.h.

typedef std::map<HashWrapper, SharedFile*>::iterator MetaDb::HWTSFIter [private]

HashWrapper To SharedFile Iterator.

Definition at line 255 of file metadb.h.

typedef std::set<MetaData*>::iterator MetaDb::LIter [private]

List Iterator.

Definition at line 186 of file metadb.h.

typedef std::multimap<std::string, SharedFile*>::iterator MetaDb::NTSFIter [private]

Name To SharedFile Iterator.

Definition at line 228 of file metadb.h.

typedef std::map<SharedFile*, MetaData*>::iterator MetaDb::SFMDIter [private]

SharedFile to MetaData Iterator.

Definition at line 201 of file metadb.h.

Constructor & Destructor Documentation

MetaDb::MetaDb ( ) [private]

Definition at line 27 of file metadb.cpp.

MetaDb::MetaDb ( MetaDb & ) [private]

MetaDb::~MetaDb ( ) [private]

Definition at line 35 of file metadb.cpp.

References m_list.

Member Function Documentation

void MetaDb::clear ( ) [private]

Clears all lists.

This is used for debugging purposes in metadata regress-test.

Definition at line 439 of file metadb.cpp.

References m_filenames, m_hashes, m_hashToSF, m_list, m_nameToSF, and m_sfToMd.

MetaData * MetaDb::find

(

SharedFile *

)

const

Find MetaData by searching with SharedFile.

Parameters:

SharedFile to search with

Returns:: Pointer to found MetaData object, or 0 if not found.

Definition at line 329 of file metadb.cpp.

References m_sfToMd.

std::vector< MetaData * > MetaDb::find

(

const std::string &

filename

)

const

Find MetaData by searching with file name.

Note that this function may return any number of MetaData objects in case the filename is ambigious.

Parameters:

filename

Name of the file to search for

Returns:: Vector containing all found entries. May be empty.

Definition at line 313 of file metadb.cpp.

References logTrace(), m_filenames, and METADB.

Here is the call graph for this function:

MetaData * MetaDb::find

(

const HashBase &

)

const

Find Metadata by searching with hash.

Parameters:

Reference to HashBase object to be searched for.

Returns:: Pointer to the MetaData object, or 0 if not found.

Definition at line 289 of file metadb.cpp.

References logTrace(), m_hashes, and METADB.

Referenced by SharedFile::findMetaData(), and tryAddHashSet().

Here is the call graph for this function:

std::vector< SharedFile * > MetaDb::findSharedFile

(

const std::string &

filename

)

const

Locate SharedFile by searching with file name.

Note that this function may return any number of SharedFiles if the file name is ambigious.

Parameters:

filename

Name of file to search for

Returns:: Vector containing all found entries. May be empty.

Definition at line 352 of file metadb.cpp.

References m_nameToSF.

SharedFile * MetaDb::findSharedFile

(

const HashBase &

)

const

Locate SharedFile by searching with hash.

Parameters:

Hash to search for

Returns:: The file being searched for, or 0 if not found.

Definition at line 338 of file metadb.cpp.

References m_hashToSF.

Referenced by SharedFile::isDuplicate().

MetaDb & MetaDb::instance ( ) [static]

This class is a Singleton.

The only instance of this class may be retrieved through this function. Note that the very first call to this function also initializes this class.

Returns:: The only instance of this class

Definition at line 41 of file metadb.cpp.

Referenced by HydraNode::cleanup(), FilesList::createDownload(), SharedFile::findMetaData(), HydraNode::initFiles(), SharedFile::isDuplicate(), FilesList::loadTempFile(), HydraNode::onEvent(), SharedFile::onHashEvent(), and SharedFile::verify().

void MetaDb::load

(

std::istream &

)

Load the MetaDb contents from input stream, adding all found entries in the stream to the database, merging duplicate entries.

Parameters:

Input stream to read from.

Definition at line 51 of file metadb.cpp.

References CHECK_THROW, logError(), logMsg(), logTrace(), OP_MDB_VERSION, push(), and TRACE_MD.

Referenced by HydraNode::initFiles().

Here is the call graph for this function:

void MetaDb::onMetaDataEvent	(	MetaData *	md,
		int	evt
	)	`[private]`

Event handler for MetaData events, called from event table.

Parameters:

	md	Event source triggering the event.
	evt	Event data specifying the kind of event.

Definition at line 369 of file metadb.cpp.

References CHECK_THROW, MD_ADDED_FILENAME, MD_ADDED_HASHSET, tryAddFileName(), and tryAddHashSet().

Referenced by push().

Here is the call graph for this function:

void MetaDb::onSharedFileEvent	(	SharedFile *	sf,
		int	evt
	)	`[private]`

Event handler for SharedFile events, called from event table.

Parameters:

	sf	SharedFile object triggering the event.
	evt	Event data specifying the kind of event.

Definition at line 387 of file metadb.cpp.

References HashSetBase::getFileHash(), HashSetBase::getFileHashTypeId(), MetaData::getFileName(), MetaData::getHashSet(), logWarning(), m_hashToSF, m_nameToSF, m_sfToMd, and SF_DESTROY.

Referenced by push(), tryAddFileName(), and tryAddHashSet().

Here is the call graph for this function:

MetaDb& MetaDb::operator= ( const MetaDb & ) [private]

void MetaDb::push	(	MetaData *	md,
		SharedFile *	sf
	)

Add a metadata object to the database, associating it with a SharedFile.

Parameters:

	md	MetaData pointer to be added to the database.
	id	SharedFile to associate with this data.

Definition at line 263 of file metadb.cpp.

References CHECK_THROW, m_sfToMd, onSharedFileEvent(), push(), tryAddFileName(), and tryAddHashSet().

Here is the call graph for this function:

void MetaDb::push

(

MetaData *

)

Add a metadata object to the database.

Parameters:

MetaData pointer to be added to the database.

Definition at line 240 of file metadb.cpp.

References CHECK_THROW, m_list, onMetaDataEvent(), tryAddFileName(), and tryAddHashSet().

Referenced by FilesList::createDownload(), load(), FilesList::loadTempFile(), SharedFile::onHashEvent(), push(), and SharedFile::verify().

Here is the call graph for this function:

void MetaDb::save

(

std::ostream &

)

const

Write the contents of the database into output stream.

Parameters:

Output stream to write to.

Definition at line 96 of file metadb.cpp.

void MetaDb::tryAddFileName	(	SharedFile *	sf,
		const std::string &	name
	)	`[private]`

Attempt to add entry to m_nameToId map.

Parameters:

	name	Filename to be added.
	id	ID of the file the name belongs to.

Definition at line 146 of file metadb.cpp.

References CHECK_THROW, logTrace(), m_nameToSF, METADB, and onSharedFileEvent().

Here is the call graph for this function:

void MetaDb::tryAddFileName	(	MetaData *	source,
		const std::string &	name
	)	`[private]`

Attempt to add entry to m_filenames list.

Parameters:

	name	Filename to be added.
	source	Source object the file name belongs to.

Definition at line 119 of file metadb.cpp.

References CHECK_THROW, logTrace(), m_filenames, and METADB.

Referenced by onMetaDataEvent(), and push().

Here is the call graph for this function:

void MetaDb::tryAddHashSet	(	SharedFile *	sf,
		const HashSetBase *	hash
	)	`[private]`

Attempt to add entry to m_hashToId map.

Parameters:

	hash	Hash to be added.
	id	ID of the hash being added.

Definition at line 206 of file metadb.cpp.

References CHECK_THROW, HashBase::getTypeId(), m_hashToSF, and onSharedFileEvent().

Here is the call graph for this function:

void MetaDb::tryAddHashSet	(	MetaData *	source,
		const HashSetBase *	hash
	)	`[private]`

Attempt to add entry to m_hashes list.

Parameters:

	hash	Hash to be added.
	source	Source object where this HashSet belongs to.

Definition at line 172 of file metadb.cpp.

References CHECK_THROW, find(), HashBase::getTypeId(), logTrace(), m_hashes, and METADB.

Referenced by onMetaDataEvent(), and push().

Here is the call graph for this function:

Friends And Related Function Documentation

std::ostream& operator<<	(	std::ostream &	o,
		const MetaDb &	md
	)	`[friend]`

Output operator for streams.

Definition at line 101 of file metadb.cpp.

void test_metadb ( ) [friend]

Member Data Documentation

std::multimap<std::string, MetaData*> MetaDb::m_filenames [private]

File Name to MetaData Map ------------------------- This map allows finding MetaData knowing file name.

The file name key here is only that - file name. No paths may be included in the name. Multiple objects may have the same file name - for this reason, this is a multi-map.

Definition at line 213 of file metadb.h.

Referenced by clear(), find(), and tryAddFileName().

std::map< CGComm::HashTypeId, std::map<HashWrapper, MetaData*> > MetaDb::m_hashes [private]

Hash To Meta Data Map --------------------- Last, but not least, a recursive map which allows us to look up MetaData objects given we know one hash.

The outer map is keyed on hash types, so there are generally only 5-6 entries in the outer map. For each of the outer map, there is an inner map, which is keyed on the actual Hash, and returns MetaData pointer. Note that we use HashWrapper wrapper class around HashBase pointer, doing virtual function calls on lookups. While it is known that virtual function calls come with some performance tradeoff, I really don't see other way of doing it. Also keep in mind that when HashSet object (contained in MetaData object) is destroyed, it would invalidate the HashWrapper pointer - make sure to not have any HashWrappers containing the HashBase objects around anymore when deleting MetaData.

Definition at line 278 of file metadb.h.

Referenced by clear(), find(), and tryAddHashSet().

std::map< CGComm::HashTypeId, std::map<HashWrapper, SharedFile*> > MetaDb::m_hashToSF [private]

Hash To SharedFile Map ---------------------- A dual map which allows us to locate SharedFiles given a hash.

The outer map is keyed on HashTypeId, so there are generally only 5-6 entries there. For each of the outer maps, there's an inner map, containing hashes of the given type. Lookups in the inner map return the SharedFile of the searched entry.

Definition at line 245 of file metadb.h.

Referenced by clear(), findSharedFile(), onSharedFileEvent(), and tryAddHashSet().

std::set<MetaData*> MetaDb::m_list [private]

Primary List ------------ Pure metadata objects.

We really cannot rely on any data existing in the contained MetaData pointer, so we can't turn this into some kind of map or anything. However, other lists contain iterators to this list for faster access - iterate directly on this list only as last resort. This list is also used to save in metadata.xml file, and is also ready for publishing to outside world.

Definition at line 184 of file metadb.h.

Referenced by clear(), push(), and ~MetaDb().

std::multimap<std::string, SharedFile*> MetaDb::m_nameToSF [private]

File Name to SharedFile Map ------------------------ This map allows searching with file name and locating the corresponding SharedFile's.

Note that this is a multimap - same file name may point to several SharedFiles.

Definition at line 226 of file metadb.h.

Referenced by clear(), findSharedFile(), onSharedFileEvent(), and tryAddFileName().

std::map<SharedFile*, MetaData*> MetaDb::m_sfToMd [private]

SharedFile To MetaData Map ---------------------- SharedFile* keyed map to locate MetaData objects by searching with SharedFile pointer.

This list is populated by FilesList (and contained) classes, and generally contains only part of the entire m_list contents - only entries which are also in FilesList are listed here.

Definition at line 199 of file metadb.h.

Referenced by clear(), find(), onSharedFileEvent(), and push().

The documentation for this class was generated from the following files:

include/hn/metadb.h
src/metadb.cpp

MetaDb Class Reference

Detailed Description

Public Member Functions

Static Public Member Functions

Private Types

Private Member Functions

Private Attributes

Friends

Classes

Member Typedef Documentation

Constructor & Destructor Documentation

Member Function Documentation

Friends And Related Function Documentation

Member Data Documentation