i need to create a function in my application to set its available memory usage. What i want to do is when the application is running, and it reaches to the set memory settings, i'll have to switch from saving to the memory to saving to a file to the local drive to avoid application hang. Is this a better way to do? What things to consider when doing this in terms of memory allocation? Hope you understand :)
Thanks,
Jepe
You might want to consider using a memory mapped file for this application.
If you write your data to a memory mapped file, you get the benefits of in-memory access to much of the data with auto-paging as needed.
You don't need to do this: let the operating system handle it automatically, it's had years of performance tweaks and improvements added over time and will generally do a much job of it.
For a longer explanation, read this from the maker of the Varnish web proxy.
You can get rough estimates with System.Diagnostics.Process or with performance counters. However, you probably should rethink this approach as you probably have better ways to tell if you should write to memory or two a disk.
First off, it's probably not total memory usage that is the issue. The problem sounds like it lives in a few, or perhaps even one, place. I would consider making that thing smarter by taking under consideration what your needs are. Then I would address the problem with design.
Maybe you need to save to disk every time but use a caching proxy to make it so that reads are from memory. Maybe you need an implementation of System.IO.Stream that delegates to a MemoryStream with predefined capacity until it gets close to that capacity and then switches over to a FileStream. Maybe it's as simple as using queuing to meter out load on a certain part of your system so that memory never becomes an issue.
Without knowing more about your specific problem, it is difficult to tell you exactly what you should do. All I can say is that predicating behavior based on memory usage is going to lead to some dicey behavior that will be difficult to test and, therefore, difficult to maintain.
That's my two cents, I suppose.
Edit:
Adding the requested class. This was not done TDD but gives you an idea of one way to solve this problem.
class UpgradingStream : Stream
{
// state pattern lives in the problem...
private abstract class InternalState
{
private readonly Stream _underlyingStream;
protected InternalState(Stream underlyingStream)
{
_underlyingStream = underlyingStream;
}
internal Stream GetUnderlyingStream()
{
return _underlyingStream;
}
// template method lives in the implementation of this state pattern
internal InternalState Seek(long offset, SeekOrigin origin, out long result)
{
result = _underlyingStream.Seek(offset, origin);
return GetNextState();
}
internal InternalState SetPosition(long value)
{
_underlyingStream.Position = value;
return GetNextState();
}
internal InternalState SetLength(long value)
{
_underlyingStream.SetLength(value);
return GetNextState();
}
internal InternalState Write(byte[] buffer, int offset, int count)
{
_underlyingStream.Write(buffer, offset, count);
return GetNextState();
}
protected abstract InternalState GetNextState();
}
private class InMemoryOnly : InternalState
{
private readonly Func<Stream> _getUpgradedStream;
private readonly int _threshold;
private InMemoryOnly(int threshold, Func<Stream> getUpgradedStream)
: base(new MemoryStream(threshold))
{
_threshold = threshold;
_getUpgradedStream = getUpgradedStream;
}
internal static InternalState GetInstance(int threshold, Func<Stream> getUpgradedStream)
{
return new InMemoryOnly(threshold, getUpgradedStream);
}
protected override InternalState GetNextState()
{
if (GetUnderlyingStream().Length > _threshold)
{
var newStream = _getUpgradedStream();
CopyStream(newStream);
return Unrestricted.GetInstance(newStream);
}
return this;
}
private void CopyStream(Stream newStream)
{
var originalPosition = GetUnderlyingStream().Position;
GetUnderlyingStream().Position = 0;
int bytesRead;
var buffer = new byte[65536];
while ((bytesRead = GetUnderlyingStream().Read(buffer, 0, buffer.Length)) != 0)
{
newStream.Write(buffer, 0, bytesRead);
}
newStream.Position = originalPosition;
}
}
private class Unrestricted : InternalState
{
private Unrestricted(Stream underlyingStream)
: base(underlyingStream)
{
}
internal static Unrestricted GetInstance(Stream stream)
{
return new Unrestricted(stream);
}
protected override InternalState GetNextState()
{
// state never changes once we are in a file or whatever
return this;
}
}
private InternalState _state;
private UpgradingStream(int threshold, Func<Stream> getMoreEfficientStream)
{
_state = InMemoryOnly.GetInstance(threshold, getMoreEfficientStream);
}
internal static Stream GetInstance(int threshold, Func<Stream> getMoreEfficientStream)
{
return new UpgradingStream(threshold, getMoreEfficientStream);
}
public override bool CanRead
{
get { return _state.GetUnderlyingStream().CanRead; }
}
public override bool CanSeek
{
get { return _state.GetUnderlyingStream().CanSeek; }
}
public override bool CanWrite
{
get { return _state.GetUnderlyingStream().CanWrite; }
}
public override void Flush()
{
_state.GetUnderlyingStream().Flush();
}
public override long Length
{
get { return _state.GetUnderlyingStream().Length; }
}
public override long Position
{
get
{
return _state.GetUnderlyingStream().Position;
}
set
{
_state = _state.SetPosition(value);
}
}
public override int Read(byte[] buffer, int offset, int count)
{
return _state.GetUnderlyingStream().Read(buffer, offset, count);
}
public override long Seek(long offset, SeekOrigin origin)
{
long result;
_state = _state.Seek(offset, origin, out result);
return result;
}
public override void SetLength(long value)
{
_state = _state.SetLength(value);
}
public override void Write(byte[] buffer, int offset, int count)
{
_state = _state.Write(buffer, offset, count);
}
public override void Close()
{
_state.GetUnderlyingStream().Close();
}
}