Object Pascal: TClientDataset Deletions

Question

I'm creating an in-memory dataset using a TClientDataset to use as a receive buffer. Adding data is great, but once I go to process it I want to be able to remove the row from the dataset. Calling delete works - sort of - the row/index is still accessible but now contains no valid information.

This makes things a bit difficult since when I'm processing this buffer it's not guaranteed that entries will be deleted in fact. I'd rather not start scanning the buffer from the first entry and skipping empty items, so is there a better way to permanently "remove" the item from the dataset? My idea was that it should work something like an actual SQL table where deleting a row doesn't leave empty records.

What's the best way to achieve this, or am I using the wrong component entirely?

Answer 1

By default client datasets mantain a "log" of changes because they are also designed to be able to send client side changes to a remote server, even if they were made in a disconnected session ("briefcase model"). Usually this log is "cleared" when you apply the changes to the remote db, and any other changes is merged with your "local" copy. Set LogChanges to False if you don't need it and wish that changes are made directly.

Answer 2

A couple of remarks regarding your code.

1. You are using an unusual way to loop through your dataset (using a counter and still using next).

2. My preferred direction when deleting would be from end to beginning.

3. You do not post your dataset after delete.

My suggestion would be to try something like this:

MyDataSet.RecNo:= 99
while not MyDataSet.Bof do
begin
  fD1 := MyDataset.FieldByName('Field1').AsInteger;
  fD2 := MyDataset.FieldByName('Field2').AsInteger;
  fD3 := MyDataset.FieldByName('Field3').AsInteger;

  if someCondition then
    MyDataset.Delete;

  MyDataSet.Post;

  MyDataset.Previous;
end;

Answer 3

There's something wrong with your code. I prepared a test application for this case, because I will face TClientDataSet in Multithreading environment in a few days. My test case application is not presenting this problem (Delphi 2010 Update 5)

I'll publish this code also in my own blog in a couple of days... for now I gave it to you now:

DFM file:

object Form2: TForm2
  Left = 0
  Top = 0
  Caption = 'Form2'
  ClientHeight = 337
  ClientWidth = 635
  Color = clBtnFace
  Font.Charset = DEFAULT_CHARSET
  Font.Color = clWindowText
  Font.Height = -11
  Font.Name = 'Tahoma'
  Font.Style = []
  OldCreateOrder = False
  OnClose = FormClose
  PixelsPerInch = 96
  TextHeight = 13
  object Memo1: TMemo
    Left = 8
    Top = 8
    Width = 257
    Height = 321
    Lines.Strings = (
      'Memo1')
    TabOrder = 0
  end
  object Button1: TButton
    Left = 271
    Top = 8
    Width = 170
    Height = 25
    Caption = 'Start'
    TabOrder = 1
    OnClick = Button1Click
  end
  object cdsTest: TClientDataSet
    Aggregates = <>
    Params = <>
    Left = 584
    Top = 32
    object cdsTestNumber: TIntegerField
      FieldName = 'Number'
    end
  end
  object tToMemo: TTimer
    Enabled = False
    Interval = 500
    OnTimer = tToMemoTimer
    Left = 376
    Top = 144
  end
end

pas file:

unit Unit2;

interface

uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, StdCtrls, DB, DBClient, SyncObjs, ExtCtrls;

type
  TWriterThread = class(TThread)
  private
    FDataSet: TClientDataSet;
    //FWriteLock: TMultiReadExclusiveWriteSynchronizer;
    FLock: TCriticalSection;
  public
    constructor Create(ADataSet: TClientDataSet; ALock: TCriticalSection);
    procedure Execute; override;
  end;

  TDeleterThread = class(TThread)
  private
    FDataSet: TClientDataSet;
    //FWriteLock: TMultiReadExclusiveWriteSynchronizer;
    FLock: TCriticalSection;
  public
    constructor Create(ADataSet: TClientDataSet; ALock: TCriticalSection);
    procedure Execute; override;
  end;

  TForm2 = class(TForm)
    cdsTest: TClientDataSet;
    Memo1: TMemo;
    cdsTestNumber: TIntegerField;
    Button1: TButton;
    tToMemo: TTimer;
    procedure Button1Click(Sender: TObject);
    procedure tToMemoTimer(Sender: TObject);
    procedure FormClose(Sender: TObject; var Action: TCloseAction);
  private
    { Private declarations }
    FLock: TCriticalSection;
    FWriterThread: TWriterThread;
    FDeleterThread: TDeleterThread;
    procedure cdsToMemo;
  public
    { Public declarations }
  end;

var
  Form2: TForm2;

implementation

{$R *.dfm}

procedure TForm2.Button1Click(Sender: TObject);
begin
  Button1.Enabled := False;
  cdsTest.CreateDataSet;
  cdsTest.LogChanges := False;
  FLock := TCriticalSection.Create;
  tToMemo.Enabled := True;
  FWriterThread := TWriterThread.Create(cdsTest, FLock);
  FDeleterThread := TDeleterThread.Create(cdsTest, FLock);
end;

{ TWriterThread }

constructor TWriterThread.Create(ADataSet: TClientDataSet;
  ALock: TCriticalSection);
begin
  inherited Create(False);
  FDataSet := ADataSet;
  FLock := ALock;
end;

procedure TWriterThread.Execute;
var
  I: Integer;
begin
  inherited;
  I := 0;
  while not Terminated do
  begin
    FLock.Enter;
    try
      Inc(I);
      FDataSet.AppendRecord([I]);
    finally
      FLock.Leave;
    end;
    Sleep(500);  //a new record aproximately each half second
  end;
end;

{ TDeleterThread }

constructor TDeleterThread.Create(ADataSet: TClientDataSet;
  ALock: TCriticalSection);
begin
  inherited Create(False);
  FDataSet := ADataSet;
  FLock := ALock;
end;

procedure TDeleterThread.Execute;
const
  MaxRecords = 100;
var
  ProcessedRecords: Integer;
begin
  inherited;
  while not Terminated do
  begin
    Sleep(3000);  //delete records aproximately every 3 seconds
    FLock.Enter;
    try
      FDataSet.First;
      ProcessedRecords := 0;
      while (not FDataSet.Eof) and (ProcessedRecords < MaxRecords) do
      begin
        Inc(ProcessedRecords);
        if Odd(FDataSet.Fields[0].AsInteger) then
          FDataSet.Delete
        else
          FDataSet.Next;
      end;
    finally
      FLock.Leave;
    end;
  end;
end;

procedure TForm2.cdsToMemo;
begin
  FLock.Enter;
  try
    Memo1.Lines.BeginUpdate;
    try
      Memo1.Lines.Clear;
      cdsTest.First;
      while not cdsTest.Eof do
      begin
        Memo1.Lines.Add(cdsTestNumber.AsString);
        cdsTest.Next;
      end;
    finally
      Memo1.Lines.EndUpdate;
    end;
  finally
    FLock.Leave;
  end;
end;

procedure TForm2.FormClose(Sender: TObject; var Action: TCloseAction);
begin
  tToMemo.Enabled := False;
  if cdsTest.Active then
  begin
    FDeleterThread.Terminate;
    FDeleterThread.WaitFor;
    FWriterThread.Terminate;
    FWriterThread.WaitFor;
  end;
end;

procedure TForm2.tToMemoTimer(Sender: TObject);
begin
  tToMemo.Enabled := False;
  cdsToMemo;
  tToMemo.Enabled := True;
end;

end.

I'll no post further explanation, because you seems well versed in multi-threading. If you have any doubt, feel free to comment with questions.

Only one thing... I was planning to use TMultiReadExclusiveWriteSynchronizer to allow better concurrence, but I have no experience in promoting ReadAccess to WriteAccess, so I used a CriticalSection to avoid the time needed to investigate right now.