tpie::file< T >::stream Class Reference

Central stream abstraction. More...

#include <tpie/file.h>

Inherits tpie::file_base::stream.

Public Types
typedef T	item_type
	Type of items stored in the stream. More...
typedef file	file_type
	Type of underlying file object. More...
enum	offset_type
	Type describing how we should interpret the offset supplied to seek. More...

Public Member Functions
	stream (file_type &file, stream_size_type offset=0)
item_type &	read_mutable ()
	Read a mutable item from the stream. More...
const item_type &	read ()
	Read an item from the stream. More...
const item_type &	read_back ()
	Read an item from the stream. More...
void	write (const item_type &item)
	Write an item to the stream. More...
template<typename IT >
void	write (const IT &start, const IT &end)
	Write several items to the stream. More...
template<typename IT >
void	read (const IT &start, const IT &end)
	Reads several items from the stream. More...
void	attach (file &f)
	Attach to the given tpie::file. If necessary, detach first. More...
void	detach ()
	Detach from a tpie::file. More...
bool	attached () const
	True if we are attached to a tpie::file. More...
memory_size_type	block_items () const
	Fetch number of items per block. More...
void	seek (stream_offset_type offset, offset_type whence=beginning)
	Moves the logical offset in the stream. More...
stream_size_type	offset () const throw ()
	Calculate the current offset in the stream. More...
bool	can_read () const throw ()
	Check if we can read an item with read(). More...
bool	can_read_back () const throw ()
	Check if we can read an item with read_back(). More...
stream_size_type	size () const throw ()
	Get the size of the file measured in items. More...

Static Public Member Functions
static memory_size_type	memory_usage (double blockFactor=1.0)
	Calculate the memory usage of a stream. More...

Protected Member Functions
block_t &	get_block ()
const block_t &	get_block () const
file_base &	get_file ()
const file_base &	get_file () const
void	update_block_core ()
void	update_vars ()
void	attach_inner (file_base &f)
	Attach to the given tpie::file. If necessary, detach first. More...
void	detach_inner ()
	Detach from a tpie::file. More...
void	write_update ()
	Call whenever the current block buffer is modified. More...
void	initialize ()
	Set up block buffers and offsets. More...
void	update_block ()
	Fetch block from disk as indicated by m_nextBlock, writing old block to disk if needed. More...

Static Protected Member Functions
static void	read_array (Stream &stream, const IT &start, const IT &end)
	Reads several items from the stream. More...
static void	write_array (Stream &stream, const IT &start, const IT &end)
	Write several items to the stream. More...

Protected Attributes
block_t *	m_block
	Current block. More...
memory_size_type	m_index
	Item index into the current block, or maxint if we don't have a block. More...
stream_size_type	m_nextBlock
	After a cross-block seek: Block index of next block, or maxint if the current block is good enough OR if we haven't read/written anything yet. More...
memory_size_type	m_nextIndex
	After a cross-block seek: Item index into next block. More...
stream_size_type	m_blockStartIndex
	The file-level item index of the first item in the current block. More...

Detailed Description

template<typename T>
class tpie::file< T >::stream

Central stream abstraction.

Conceptually compatible with Reading and writing file streams.

Definition at line 71 of file file.h.

Member Typedef Documentation

file_type

template<typename T >

typedef file tpie::file< T >::stream::file_type

Type of underlying file object.

Definition at line 76 of file file.h.

item_type

template<typename T >

typedef T tpie::file< T >::stream::item_type

Type of items stored in the stream.

Definition at line 74 of file file.h.

Member Enumeration Documentation

offset_type

enum tpie::stream_crtp::offset_type

inherited

Type describing how we should interpret the offset supplied to seek.

Definition at line 39 of file stream_crtp.h.

                     {
        beginning,
        end,
        current
    };

Member Function Documentation

attach()

template<typename T >

void tpie::file< T >::stream::attach ( file &  f )

inline

Attach to the given tpie::file. If necessary, detach first.

Definition at line 190 of file file.h.

                                     {
            attach_inner(f);
        }

References tpie::file_base::stream::attach_inner().

attach_inner()

void tpie::file_base::stream::attach_inner ( file_base &  f )

protectedinherited

Attach to the given tpie::file. If necessary, detach first.

Referenced by tpie::file< T >::stream::attach().

attached()

bool tpie::file_base::stream::attached ( ) const

inlineinherited

True if we are attached to a tpie::file.

Definition at line 113 of file file_base.h.

{ return 0 != m_file; }

block_items()

memory_size_type tpie::file_base::stream::block_items ( ) const

inlineinherited

Fetch number of items per block.

Definition at line 130 of file file_base.h.

{return get_file().m_blockItems;}

Referenced by tpie::file< T >::stream::write().

can_read()

bool tpie::stream_crtp< stream >::can_read throw ( )

inlineinherited

Check if we can read an item with read().

This is logically equivalent to:

return offset() < size();

but it might be faster.

Returns

Whether or not we can read more items from the stream.

Definition at line 109 of file stream_crtp.h.

                                         {
        assert(self().get_file().is_open());
        if (m_index < self().get_block().size ) return true;
        return offset() < self().size();
    }

can_read_back()

bool tpie::stream_crtp< stream >::can_read_back throw ( )

inlineinherited

Check if we can read an item with read_back().

Returns

Whether or not we can read an item with read_back().

Definition at line 120 of file stream_crtp.h.

                                              {
        assert(self().get_file().is_open());
        if (m_nextBlock == std::numeric_limits<stream_size_type>::max())
            return m_index > 0 || m_blockStartIndex > 0;
        else
            return m_nextIndex > 0 || m_nextBlock > 0;
    }

detach()

template<typename T >

void tpie::file< T >::stream::detach ( )

inline

Detach from a tpie::file.

Definition at line 197 of file file.h.

                             {
            detach_inner();
        }

References tpie::file_base::stream::detach_inner().

detach_inner()

void tpie::file_base::stream::detach_inner ( )

protectedinherited

Detach from a tpie::file.

Referenced by tpie::file< T >::stream::detach().

initialize()

void tpie::file_base::stream::initialize ( )

inlineprotectedinherited

Set up block buffers and offsets.

Definition at line 170 of file file_base.h.

                                 {
            if (m_block != &get_file().m_emptyBlock) get_file().free_block(m_block);
            p_t::initialize();
            m_block = &get_file().m_emptyBlock;
        }

memory_usage()

template<typename T >

static memory_size_type tpie::file< T >::stream::memory_usage ( double  blockFactor = 1.0 )

inlinestatic

Calculate the memory usage of a stream.

Definition at line 84 of file file.h.

                                                                            {
            return sizeof(stream) + block_size(blockFactor) +  sizeof(block_t);
        }

References tpie::file_base_crtp< file_base >::block_size().

offset()

stream_size_type tpie::stream_crtp< stream >::offset throw ( )

inlineinherited

Calculate the current offset in the stream.

Returns

The current offset in the stream

Definition at line 91 of file stream_crtp.h.

                                                   {
        assert(self().get_file().is_open());
        if (m_nextBlock == std::numeric_limits<stream_size_type>::max())
            return m_index + m_blockStartIndex;
        return m_nextIndex + m_nextBlock * self().get_file().block_items();
    }

read() [1/2]

template<typename T >

const item_type& tpie::file< T >::stream::read ( )

inline

Read an item from the stream.

Read current item from the stream, and increment the offset by one item.

This will throw an end_of_stream_exception if there are no more items left in the stream.

To ensure that no exception is thrown, check that can_read() returns true.

Returns

The item read from the stream.

Definition at line 126 of file file.h.

                                        {
            return read_mutable();
        }

References tpie::file< T >::stream::read_mutable().

Referenced by tpie::file< T >::stream::read_back().

read() [2/2]

template<typename T >

template<typename IT >

void tpie::file< T >::stream::read ( const IT &  start, const IT &  end  )

inline

Reads several items from the stream.

Implementation note: If your iterator type is efficiently copyable with std::copy, then this will also read efficiently from the internal TPIE buffer.

Template Parameters

IT	The type of Random Access Iterators used to supply the items.

Parameters

start	Iterator to the first spot to write to.
end	Iterator past the last spot to write to.

Exceptions

end_of_stream_exception

If there are not enough elements in the stream to fill all the spots between start and end.

See also

file_stream<T>::read(const IT & start, const IT & end)

Definition at line 182 of file file.h.

                                                           {
            assert(get_file().is_open());
            read_array(*this, start, end);
        }

References tpie::file_base_crtp< file_base >::is_open(), and tpie::stream_crtp< stream >::read_array().

read_array()

static void tpie::stream_crtp< stream >::read_array ( Stream &  stream, const IT &  start, const IT &  end  )

inlinestaticprotectedinherited

Reads several items from the stream.

Implementation note: If your iterator type is efficiently copyable with std::copy, then this will also read efficiently from the internal TPIE buffer.

Template Parameters

IT	The type of Random Access Iterators used to supply the items.

Parameters

start	Iterator to the first spot to write to.
end	Iterator past the last spot to write to.

Exceptions

end_of_stream_exception

If there are not enough elements in the stream to fill all the spots between start and end.

Definition at line 166 of file stream_crtp.h.

                                                                                     {
        typedef typename Stream::item_type T;
        IT i = start;
        while (i != end) {
            if (stream.m_index >= stream.block_items()) {
                // check to make sure we have enough items in the stream
                stream_size_type offs = stream.offset();
                if (offs >= stream.size()
                    || offs + (end-i) > stream.size()) {
 
                    throw end_of_stream_exception();
                }
 
                // fetch next block from disk
                stream.update_block();
            }
 
            T * src = reinterpret_cast<T*>(stream.get_block().data) + stream.m_index;
 
            // either read the rest of the block or until `end'
            memory_size_type count = std::min(stream.block_items()-stream.m_index, static_cast<memory_size_type>(end-i));
 
            std::copy(src, src + count, i);
 
            // advance output iterator
            i += count;
 
            // advance input position
            stream.m_index += count;
        }
    }

read_back()

template<typename T >

const item_type& tpie::file< T >::stream::read_back ( )

inline

Read an item from the stream.

Decrement the offset by one, and read current item from the stream.

This will throw an end_of_stream_exception if there are no more items left in the stream.

To ensure that no exception is thrown, check that can_read_back() returns true.

Returns

The item read from the stream.

Definition at line 143 of file file.h.

                                             {
            assert(get_file().is_open());
            seek(-1, current);
            const item_type & i = read();
            seek(-1, current);
            return i;
        }

References tpie::file_base_crtp< file_base >::is_open(), tpie::file< T >::stream::read(), and tpie::stream_crtp< stream >::seek().

read_mutable()

template<typename T >

item_type& tpie::file< T >::stream::read_mutable ( )

inline

Read a mutable item from the stream.

Don't use this method. Instead, use file<T>::stream::read().

Read current item from the stream, and increment the offset by one item.

This will throw an end_of_stream_exception if there are no more items left in the stream.

To ensure that no exception is thrown, check that can_read() returns true.

Returns

The item read from the stream.

Definition at line 101 of file file.h.

                                          {
            assert(get_file().is_open());
            if (m_index >= m_block->size) {
                update_block();
                if (offset() >= get_file().size()) {
                    throw end_of_stream_exception();
                }
            }
            return reinterpret_cast<T*>(m_block->data)[m_index++];
        }

References tpie::file_base_crtp< file_base >::is_open(), tpie::file_base::stream::m_block, tpie::stream_crtp< stream >::m_index, tpie::stream_crtp< stream >::offset(), tpie::stream_crtp< stream >::size(), and tpie::stream_crtp< stream >::update_block().

Referenced by tpie::file< T >::stream::read().

seek()

void tpie::stream_crtp< stream >::seek ( stream_offset_type  offset, offset_type  whence = beginning  )

inlineinherited

Moves the logical offset in the stream.

Parameters

offset	Where to move the logical offset to.
whence	Move the offset relative to what.

Definition at line 51 of file stream_crtp.h.

                                                                              {
        assert(self().get_file().is_open());
        if (whence == end)
            offset += self().size();
        else if (whence == current) {
            // are we seeking into the current block?
            if (offset >= 0 || static_cast<stream_size_type>(-offset) <= m_index) {
                stream_size_type new_index = static_cast<stream_offset_type>(offset+m_index);
 
                if (new_index < self().get_file().block_items()) {
                    self().update_vars();
                    m_index = static_cast<memory_size_type>(new_index);
                    return;
                }
            }
 
            offset += self().offset();
        }
        if (0 > offset || (stream_size_type)offset > self().size())
            throw io_exception("Tried to seek out of file");
        self().update_vars();
        stream_size_type b = static_cast<stream_size_type>(offset) / self().get_file().block_items();
        m_index = static_cast<memory_size_type>(offset - b* self().get_file().block_items());
        if (b == self().get_block().number) {
            m_nextBlock = std::numeric_limits<stream_size_type>::max();
            m_nextIndex = std::numeric_limits<memory_size_type>::max();
            assert(self().offset() == (stream_size_type)offset);
            return;
        }
        m_nextBlock = b;
        m_nextIndex = m_index;
        m_index = std::numeric_limits<memory_size_type>::max();
        assert(self().offset() == (stream_size_type)offset);
    }

size()

stream_size_type tpie::stream_crtp< stream >::size throw ( )

inlineinherited

Get the size of the file measured in items.

Returns

The number of items in the file.

Definition at line 133 of file stream_crtp.h.

                                                 {
        // XXX update_vars changes internal state in a way that is not visible
        // through the class interface.
        // therefore, a const_cast is warranted.
        const_cast<child_t&>(self()).update_vars();
        return self().get_file().file_size();
    }

update_block()

void tpie::stream_crtp< stream >::update_block

protectedinherited

Fetch block from disk as indicated by m_nextBlock, writing old block to disk if needed.

Update m_block, m_index, m_nextBlock and m_nextIndex. If m_nextBlock is maxint, use next block is the one numbered m_block->number+1. m_index is updated with the value of m_nextIndex.

write() [1/2]

template<typename T >

template<typename IT >

void tpie::file< T >::stream::write ( const IT &  start, const IT &  end  )

inline

Write several items to the stream.

Implementation note: If your iterator type is efficiently copyable with std::copy, then this will also write efficiently into the internal TPIE buffer.

Template Parameters

IT	The type of Random Access Iterators used to supply the items.

Parameters

start	Iterator to the first item to write.
end	Iterator past the last item to write.

See also

file_stream<T>::write(const IT & start, const IT & end)

Definition at line 172 of file file.h.

                                                            {
            assert(get_file().is_open());
            write_array(*this, start, end);
        }

References tpie::file_base_crtp< file_base >::is_open(), and tpie::stream_crtp< stream >::write_array().

write() [2/2]

template<typename T >

void tpie::file< T >::stream::write ( const item_type &  item )

inline

Write an item to the stream.

Parameters

item	The item to write to the stream.

Definition at line 156 of file file.h.

                                                 {
            assert(get_file().is_open());
#ifndef NDEBUG
            if (!get_file().is_writable())
                throw io_exception("Cannot write to read only stream");
#endif
            if (m_index >= block_items()) update_block();
            reinterpret_cast<T*>(m_block->data)[m_index++] = item;
            write_update();
        }

References tpie::file_base::stream::block_items(), tpie::file_base_crtp< file_base >::is_open(), tpie::file_base_crtp< file_base >::is_writable(), tpie::file_base::stream::m_block, tpie::stream_crtp< stream >::m_index, tpie::stream_crtp< stream >::update_block(), and tpie::file_base::stream::write_update().

write_array()

static void tpie::stream_crtp< stream >::write_array ( Stream &  stream, const IT &  start, const IT &  end  )

inlinestaticprotectedinherited

Write several items to the stream.

Implementation note: If your iterator type is efficiently copyable with std::copy, then this will also write efficiently into the internal TPIE buffer.

Template Parameters

IT	The type of Random Access Iterators used to supply the items.

Parameters

start	Iterator to the first item to write.
end	Iterator past the last item to write.

Definition at line 212 of file stream_crtp.h.

                                                                                      {
        typedef typename Stream::item_type T;
        IT i = start;
        while (i != end) {
            if (stream.m_index >= stream.block_items()) stream.update_block();
 
            size_t streamRemaining = end - i;
            size_t blockRemaining = stream.block_items()-stream.m_index;
 
            IT till = (blockRemaining < streamRemaining) ? (i + blockRemaining) : end;
 
            T * dest = reinterpret_cast<T*>(stream.get_block().data) + stream.m_index;
 
            std::copy(i, till, dest);
 
            stream.m_index += till - i;
            stream.write_update();
            i = till;
        }
    }

write_update()

void tpie::file_base::stream::write_update ( )

inlineprotectedinherited

Call whenever the current block buffer is modified.

Since we support multiple streams per block, we must always keep m_block->size updated when m_block is the trailing block (or the only block) in the file. For the same reasons we keep m_file->m_size updated.

Definition at line 140 of file file_base.h.

                                   {
            m_block->dirty = true;
            m_block->size = std::max(m_block->size, m_index);
            get_file().update_size(static_cast<stream_size_type>(m_index)+m_blockStartIndex);
        }

Referenced by tpie::file< T >::stream::write().

Member Data Documentation

m_block

block_t* tpie::file_base::stream::m_block

protectedinherited

Current block.

May be equal to &m_file->m_emptyBlock to indicate no current block.

Definition at line 107 of file file_base.h.

Referenced by tpie::file< T >::stream::read_mutable(), and tpie::file< T >::stream::write().

m_blockStartIndex

stream_size_type tpie::stream_crtp< stream >::m_blockStartIndex

protectedinherited

The file-level item index of the first item in the current block.

When m_block is not the null block, this should be equal to m_block->number * block_items().

Definition at line 256 of file stream_crtp.h.

m_index

memory_size_type tpie::stream_crtp< stream >::m_index

protectedinherited

Item index into the current block, or maxint if we don't have a block.

Definition at line 245 of file stream_crtp.h.

m_nextBlock

stream_size_type tpie::stream_crtp< stream >::m_nextBlock

protectedinherited

After a cross-block seek: Block index of next block, or maxint if the current block is good enough OR if we haven't read/written anything yet.

Definition at line 249 of file stream_crtp.h.

m_nextIndex

memory_size_type tpie::stream_crtp< stream >::m_nextIndex

protectedinherited

After a cross-block seek: Item index into next block.

Otherwise, maxint as with m_nextBlock.

Definition at line 252 of file stream_crtp.h.

---

The documentation for this class was generated from the following file:

• file.h