Stream in file. We support multiple streams per file. More...

#include <tpie/file_base.h>

Public Types
enum	offset_type
	Type describing how we should interpret the offset supplied to seek. More...

Public Member Functions
bool	attached () const
	True if we are attached to a tpie::file. More...

memory_size_type	block_items () const
	Fetch number of items per block. More...

	stream (file_base &file, stream_size_type offset=0)
	Create a stream associated with the given file. More...

void	seek (stream_offset_type offset, offset_type whence=beginning)
	Moves the logical offset in the stream. More...

stream_size_type	offset () const throw ()
	Calculate the current offset in the stream. More...

bool	can_read () const throw ()
	Check if we can read an item with read(). More...

bool	can_read_back () const throw ()
	Check if we can read an item with read_back(). More...

stream_size_type	size () const throw ()
	Get the size of the file measured in items. More...

Protected Member Functions
block_t &	get_block ()

const block_t &	get_block () const

file_base &	get_file ()

const file_base &	get_file () const

void	update_block_core ()

void	update_vars ()

void	attach_inner (file_base &f)
	Attach to the given tpie::file. If necessary, detach first. More...

void	detach_inner ()
	Detach from a tpie::file. More...

void	write_update ()
	Call whenever the current block buffer is modified. More...

void	initialize ()
	Set up block buffers and offsets. More...

void	update_block ()
	Fetch block from disk as indicated by m_nextBlock, writing old block to disk if needed. More...

Static Protected Member Functions
static void	read_array (Stream &stream, const IT &start, const IT &end)
	Reads several items from the stream. More...

static void	write_array (Stream &stream, const IT &start, const IT &end)
	Write several items to the stream. More...

Protected Attributes
block_t *	m_block
	Current block. More...

memory_size_type	m_index
	Item index into the current block, or maxint if we don't have a block. More...

stream_size_type	m_nextBlock
	After a cross-block seek: Block index of next block, or maxint if the current block is good enough OR if we haven't read/written anything yet. More...

memory_size_type	m_nextIndex
	After a cross-block seek: Item index into next block. More...

stream_size_type	m_blockStartIndex
	The file-level item index of the first item in the current block. More...

Friends
class	stream_crtp< stream >

Detailed Description

Stream in file. We support multiple streams per file.

Definition at line 86 of file file_base.h.

Member Enumeration Documentation

◆ offset_type

enum tpie::stream_crtp::offset_type

inherited

Type describing how we should interpret the offset supplied to seek.

Definition at line 39 of file stream_crtp.h.

                      {
         beginning,
         end,
         current
     };

Constructor & Destructor Documentation

◆ stream()

tpie::file_base::stream::stream	(	file_base &	file,
		stream_size_type	offset = `0`
	)

Create a stream associated with the given file.

Parameters

file	The file to associate with this stream.
offset	The file-level item offset to seek to.

Member Function Documentation

◆ attach_inner()

void tpie::file_base::stream::attach_inner ( file_base & f )

protected

Attach to the given tpie::file. If necessary, detach first.

Referenced by tpie::file< T >::stream::attach().

◆ attached()

bool tpie::file_base::stream::attached ( ) const

inline

True if we are attached to a tpie::file.

Definition at line 113 of file file_base.h.

113 { return 0 != m_file; }

◆ block_items()

memory_size_type tpie::file_base::stream::block_items ( ) const

inline

Fetch number of items per block.

Definition at line 130 of file file_base.h.

130 {return get_file().m_blockItems;}

Referenced by tpie::file< T >::stream::write().

◆ can_read()

bool tpie::stream_crtp< stream >::can_read	(	)	const
throw	(
	)

inlineinherited

Check if we can read an item with read().

This is logically equivalent to:

return offset() < size();

but it might be faster.

Returns: Whether or not we can read more items from the stream.

Definition at line 109 of file stream_crtp.h.

                                          {
         assert(self().get_file().is_open());
         if (m_index < self().get_block().size ) return true;
         return offset() < self().size();
     }

◆ can_read_back()

bool tpie::stream_crtp< stream >::can_read_back	(	)	const
throw	(
	)

inlineinherited

Check if we can read an item with read_back().

Returns: Whether or not we can read an item with read_back().

Definition at line 120 of file stream_crtp.h.

                                               {
         assert(self().get_file().is_open());
         if (m_nextBlock == std::numeric_limits<stream_size_type>::max())
             return m_index > 0 || m_blockStartIndex > 0;
         else
             return m_nextIndex > 0 || m_nextBlock > 0;
     }

◆ detach_inner()

void tpie::file_base::stream::detach_inner ( )

protected

Detach from a tpie::file.

Referenced by tpie::file< T >::stream::detach().

◆ initialize()

void tpie::file_base::stream::initialize ( )

inlineprotected

Set up block buffers and offsets.

Definition at line 170 of file file_base.h.

                                  {
             if (m_block != &get_file().m_emptyBlock) get_file().free_block(m_block);
             p_t::initialize();
             m_block = &get_file().m_emptyBlock;
         }

◆ offset()

stream_size_type tpie::stream_crtp< stream >::offset	(	)	const
throw	(
	)

inlineinherited

Calculate the current offset in the stream.

Returns: The current offset in the stream

Definition at line 91 of file stream_crtp.h.

                                                    {
         assert(self().get_file().is_open());
         if (m_nextBlock == std::numeric_limits<stream_size_type>::max())
             return m_index + m_blockStartIndex;
         return m_nextIndex + m_nextBlock * self().get_file().block_items();
     }

◆ read_array()

static void tpie::stream_crtp< stream >::read_array	(	Stream &	stream,
		const IT &	start,
		const IT &	end
	)

inlinestaticprotectedinherited

Reads several items from the stream.

Implementation note: If your iterator type is efficiently copyable with std::copy, then this will also read efficiently from the internal TPIE buffer.

Template Parameters

IT	The type of Random Access Iterators used to supply the items.

Parameters

start	Iterator to the first spot to write to.
end	Iterator past the last spot to write to.

Exceptions

end_of_stream_exception If there are not enough elements in the stream to fill all the spots between start and end.

Definition at line 166 of file stream_crtp.h.

                                                                                      {
         typedef typename Stream::item_type T;
         IT i = start;
         while (i != end) {
             if (stream.m_index >= stream.block_items()) {
                 // check to make sure we have enough items in the stream
                 stream_size_type offs = stream.offset();
                 if (offs >= stream.size()
                     || offs + (end-i) > stream.size()) {
  
                     throw end_of_stream_exception();
                 }
  
                 // fetch next block from disk
                 stream.update_block();
             }
  
             T * src = reinterpret_cast<T*>(stream.get_block().data) + stream.m_index;
  
             // either read the rest of the block or until `end'
             memory_size_type count = std::min(stream.block_items()-stream.m_index, static_cast<memory_size_type>(end-i));
  
             std::copy(src, src + count, i);
  
             // advance output iterator
             i += count;
  
             // advance input position
             stream.m_index += count;
         }
     }

◆ seek()

void tpie::stream_crtp< stream >::seek	(	stream_offset_type	offset,
		offset_type	whence = `beginning`
	)

inlineinherited

Moves the logical offset in the stream.

Parameters

offset	Where to move the logical offset to.
whence	Move the offset relative to what.

Definition at line 51 of file stream_crtp.h.

                                                                               {
         assert(self().get_file().is_open());
         if (whence == end)
             offset += self().size();
         else if (whence == current) {
             // are we seeking into the current block?
             if (offset >= 0 || static_cast<stream_size_type>(-offset) <= m_index) {
                 stream_size_type new_index = static_cast<stream_offset_type>(offset+m_index);
  
                 if (new_index < self().get_file().block_items()) {
                     self().update_vars();
                     m_index = static_cast<memory_size_type>(new_index);
                     return;
                 }
             }
  
             offset += self().offset();
         }
         if (0 > offset || (stream_size_type)offset > self().size())
             throw io_exception("Tried to seek out of file");
         self().update_vars();
         stream_size_type b = static_cast<stream_size_type>(offset) / self().get_file().block_items();
         m_index = static_cast<memory_size_type>(offset - b* self().get_file().block_items());
         if (b == self().get_block().number) {
             m_nextBlock = std::numeric_limits<stream_size_type>::max();
             m_nextIndex = std::numeric_limits<memory_size_type>::max();
             assert(self().offset() == (stream_size_type)offset);
             return;
         }
         m_nextBlock = b;
         m_nextIndex = m_index;
         m_index = std::numeric_limits<memory_size_type>::max();
         assert(self().offset() == (stream_size_type)offset);
     }

◆ size()

stream_size_type tpie::stream_crtp< stream >::size	(	)	const
throw	(
	)

inlineinherited

Get the size of the file measured in items.

Returns: The number of items in the file.

Definition at line 133 of file stream_crtp.h.

                                                  {
         // XXX update_vars changes internal state in a way that is not visible
         // through the class interface.
         // therefore, a const_cast is warranted.
         const_cast<child_t&>(self()).update_vars();
         return self().get_file().file_size();
     }

◆ update_block()

void tpie::stream_crtp< stream >::update_block ( )

protectedinherited

Fetch block from disk as indicated by m_nextBlock, writing old block to disk if needed.

Update m_block, m_index, m_nextBlock and m_nextIndex. If m_nextBlock is maxint, use next block is the one numbered m_block->number+1. m_index is updated with the value of m_nextIndex.

◆ write_array()

static void tpie::stream_crtp< stream >::write_array	(	Stream &	stream,
		const IT &	start,
		const IT &	end
	)

inlinestaticprotectedinherited

Write several items to the stream.

Implementation note: If your iterator type is efficiently copyable with std::copy, then this will also write efficiently into the internal TPIE buffer.

Template Parameters

IT	The type of Random Access Iterators used to supply the items.

Parameters

start	Iterator to the first item to write.
end	Iterator past the last item to write.

Definition at line 212 of file stream_crtp.h.

                                                                                       {
         typedef typename Stream::item_type T;
         IT i = start;
         while (i != end) {
             if (stream.m_index >= stream.block_items()) stream.update_block();
  
             size_t streamRemaining = end - i;
             size_t blockRemaining = stream.block_items()-stream.m_index;
  
             IT till = (blockRemaining < streamRemaining) ? (i + blockRemaining) : end;
  
             T * dest = reinterpret_cast<T*>(stream.get_block().data) + stream.m_index;
  
             std::copy(i, till, dest);
  
             stream.m_index += till - i;
             stream.write_update();
             i = till;
         }
     }

◆ write_update()

void tpie::file_base::stream::write_update ( )

inlineprotected

Call whenever the current block buffer is modified.

Since we support multiple streams per block, we must always keep m_block->size updated when m_block is the trailing block (or the only block) in the file. For the same reasons we keep m_file->m_size updated.

Definition at line 140 of file file_base.h.

                                    {
             m_block->dirty = true;
             m_block->size = std::max(m_block->size, m_index);
             get_file().update_size(static_cast<stream_size_type>(m_index)+m_blockStartIndex);
         }

Referenced by tpie::file< T >::stream::write().

Member Data Documentation

◆ m_block

block_t* tpie::file_base::stream::m_block

protected

Current block.

May be equal to &m_file->m_emptyBlock to indicate no current block.

Definition at line 107 of file file_base.h.

Referenced by tpie::file< T >::stream::read_mutable(), and tpie::file< T >::stream::write().

◆ m_blockStartIndex

stream_size_type tpie::stream_crtp< stream >::m_blockStartIndex

protectedinherited

The file-level item index of the first item in the current block.

When m_block is not the null block, this should be equal to m_block->number * block_items().

Definition at line 256 of file stream_crtp.h.

◆ m_index

memory_size_type tpie::stream_crtp< stream >::m_index

protectedinherited

Item index into the current block, or maxint if we don't have a block.

Definition at line 245 of file stream_crtp.h.

◆ m_nextBlock

stream_size_type tpie::stream_crtp< stream >::m_nextBlock

protectedinherited

After a cross-block seek: Block index of next block, or maxint if the current block is good enough OR if we haven't read/written anything yet.

Definition at line 249 of file stream_crtp.h.

◆ m_nextIndex

memory_size_type tpie::stream_crtp< stream >::m_nextIndex

protectedinherited

After a cross-block seek: Item index into next block.

Otherwise, maxint as with m_nextBlock.

Definition at line 252 of file stream_crtp.h.

The documentation for this class was generated from the following file:

file_base.h

TPIE

Public Types

Public Member Functions

Protected Member Functions

Static Protected Member Functions

Protected Attributes

Friends

Detailed Description

Member Enumeration Documentation

◆ offset_type

Constructor & Destructor Documentation

◆ stream()

Member Function Documentation

◆ attach_inner()

◆ attached()

◆ block_items()

◆ can_read()

◆ can_read_back()

◆ detach_inner()

◆ initialize()

◆ offset()

◆ read_array()

◆ seek()

◆ size()

◆ update_block()

◆ write_array()

◆ write_update()

Member Data Documentation

◆ m_block

◆ m_blockStartIndex

◆ m_index

◆ m_nextBlock

◆ m_nextIndex