Suggetions for overcoming a flash waveform performance problem

Question

Hi,

My question is sort of a follow on from this question below but I thought it'd be best to start a new question.

http://stackoverflow.com/questions/3211755/help-converting-this-as3-code-to-pixel-bender-code

So I have a waveform drawing implementation in flash that extracts pcm data from an mp3 and draws a waveform at multiple zoom levels. It works fine until you begin dealing with tracks longer than a couple of minutes and then it is just unacceptedly too slow. So I'm after a faster solution.

I first thought to try and use pixel bender because I heard it is very quick at calculations. This is true but as illustrated in the link posted above the lack of for loops makes the technology an unsuitable candidate.

So now I'm thinking what would be the best avenue too purse as an alternative.

Would writing my waveform calculation code in c and using alchemy give me a perceiveable speed and performance improvement? Its essentially still just as3 running right?

Would I be better off using some sort of server sider script? Maybe passing the pcm data as a bytearray to a php script and getting the script to return the plotting data?

Can you think of any other techniques that I could try to look into to?

EDIT:>>

What I try to do in as3 is

Extract a mp3s audio data to a byte array Determine the virtual number of pixels to plot at maximum zoom level (800 pixels = 10 seconds of a track at 2x zoom for example)

So if my number of samples = 33315840 and the maximum number of virtual pixels = 1600 (scrolling required because my viewport is 800 pixels wide) then my chunk size = 33315840/1600 =20822

So for every 20822 samples in hte bytearray I find the maximum value and use that value as my plotting value

EDIT 2

@backtodos

I had a think about the points you where making and it does seem wiser than this whole max business.

I've done up the following code which I think represents what you were talking about. One problem is that if I change my resolution to anything greater than 800 I get very tall line and very short ones which looks very odd. At 800 its a very nice waveform!

If my resolution exceeds 1600 I get errors thrown when I ttry to draw the rect saying that a parameter was invalid.

I think I'm a little confused about you zooming strategy. You say "If you zoom in, you redraw the wave form with a duplicate resolution". I don't really get what you mean my that? I know I will only ever be drawing 800 pixels at a time. Lets say I do a 2x zoom, would that mean that I double the amount of reads from by sample source and have 1600 plotting points but only draw 800 of them at a time based on the scroll position?

        private function calculatePlottingData():void
        {

            this._container=new UIComponent()
            drawon=new MovieClip()
            _container.addChild(drawon)
            addChild(_container)

            var spriteSize:Number;
            spriteSize=800; // my resolution

            //ba is a bytearray filled with my extracted audio data
            //blocksize helps me determine where in the bytearray 
            //to read a sample value from for plotting
            blocksize=Math.floor(ba.length / spriteSize);

            var tstart:Number=getTimer()
            var l:Number
            var r:Number
            var la:Number
            var ra:Number
            var readpoint:int=0;
            var val:Number;

            for (var i:int=0; i < spriteSize; i++)
            {
                readpoint=i * blocksize


                ba.position=readpoint;

                //read teh left and right sample
                la=ba.readFloat();
                ra=ba.readFloat();

                val=(la + ra) / 2;

                plottingvalues[i]=val

                drawon.graphics.beginFill(0xff0000)

            }
            var tend:Number=getTimer() - tstart;
            trace("time taken=" + tend / 1000)

            draw()
        }

        private function draw():void
        {
            var val:Number
            for (var i:int=0; i < _viewportwidth; i++)
            {
                drawon.graphics.beginFill(0xffffff)
                val=plottingvalues[i];
                drawon.graphics.drawRect(i, 50, 1, val * 50)
            }
        }

EDIT 3 > Improved code based on getting only ever getting 800 plotting points this code is looking to be quick enough for my purposes. It saves a lot of processing using @backtodos's suggestion of sticking to a resolution.

    package drawing
{
    import flash.display.Bitmap;
    import flash.display.BitmapData;
    import flash.display.MovieClip;
    import flash.events.MouseEvent;
    import flash.geom.Rectangle;
    import flash.media.Sound;
    import flash.utils.ByteArray;
    import flash.utils.getTimer;

    import mx.core.Application;

    public class WaveformView extends MovieClip
    {
        private var _sound:Sound
        private var _resolution:Number
        private var _quality:int=1;
        private var _buffersize:int;
        private var _ba:ByteArray=new ByteArray()
        private var _step:Number
        private var _zoomby:int=1;
        private var _zoomfactor:int=2;
        private var _blocksize:int;
        private var _scrollbar:Bitmap
        private var _scrollthumb:MovieClip


        private var _relativeThumbPosition:Number=0
        private var _sliderRange:int
        private var _sliderCenterMinLeft:int
        private var _sliderCenterMax_Right:int;
        public var _slider_rel_pos:Number=0;
        private var _sliderCenter:Number=0;

        public function WaveformView(snd:Sound, res:Number=800, quality:int=2, buffersize:int=2048)
        {
            super();
            this._sound=snd;
            this._resolution=res;
            this._quality=quality; // not implemented yet
            this._buffersize=buffersize;
            initScrollBar();
            getPCM()
            drawwaveform()

        }

        private function initScrollBar():void
        {
            var sbbmd:BitmapData=new BitmapData(_resolution, 20, false, 0xcccccc)
            _scrollbar=new Bitmap(sbbmd)
            _scrollbar.y=120
            addChild(_scrollbar)

            sbbmd=new BitmapData(_resolution, 16, false, 0xeeeeee)
            _scrollthumb=new MovieClip()
            _scrollthumb.graphics.beginFill(0xff0000)
            _scrollthumb.graphics.drawRect(0, 0, _resolution, 10)
            _scrollthumb.graphics.endFill()
            _scrollthumb.y=125
            addChild(_scrollthumb)
            _scrollthumb.buttonMode=true
            _scrollthumb.addEventListener(MouseEvent.MOUSE_DOWN, beginthumbdrag)

        }

        private function beginthumbdrag(e:MouseEvent):void
        {
            _scrollthumb.startDrag(false, new Rectangle(_scrollbar.x, _scrollbar.y + 5, _scrollbar.width - (_scrollthumb.width), 0))
            _scrollthumb.addEventListener(MouseEvent.MOUSE_MOVE, shuttleMoving, false, 0, true);
            _scrollthumb.addEventListener(MouseEvent.MOUSE_UP, endthumbdrag, false, 0, true);
        }

        private function endthumbdrag(e:MouseEvent):void
        {
            _scrollthumb.stopDrag();
            e.updateAfterEvent();
            _scrollthumb.removeEventListener(MouseEvent.MOUSE_MOVE, shuttleMoving);
        }

        private function shuttleMoving(e:MouseEvent):void
        {
            calculateShuttleRelativePosition();
            drawwaveform()
            e.updateAfterEvent()
        }

        private function calculateShuttleRelativePosition():void
        {

            var _x:Number=_scrollthumb.x
            _sliderCenter=_x + (_scrollthumb.width / 2)
            _sliderRange=_scrollbar.width - _scrollthumb.width;
            _slider_rel_pos=(_sliderCenter - _sliderCenterMinLeft) / _sliderRange

        }

        public function getPCM():void
        {
            var len:int=_sound.length * 44.1
            _sound.extract(_ba, len)
            this._blocksize=_ba.length / (_resolution * _zoomby)
            shaveblocksize()
        }

        public function zoomin():void
        {
            if (this._zoomby < 16)
            {
                this._zoomby*=2
                resizeThumb()
                drawwaveform()
            }
        }

        public function zoomout():void
        {
            if (_zoomby >= 2)
            {
                this._zoomby/=2
                resizeThumb()

                drawwaveform()
            }
        }

        private function resizeThumb():void
        {
            _scrollthumb.width=_scrollbar.width / _zoomby
            if (_scrollthumb.width == _resolution)
            {
                _slider_rel_pos=0
            }
            _sliderCenterMinLeft=_scrollthumb.width / 2;
            _sliderCenterMax_Right=_scrollbar.width - (_scrollthumb.width / 2);
            _sliderRange=_scrollbar.width - _scrollthumb.width;
            _sliderRange=_scrollbar.width - _scrollthumb.width;
            _scrollthumb.x=(_slider_rel_pos * _sliderRange)
        }


        public function drawwaveform():void
        {
            var starttime:Number=getTimer()

            var readposition:int
            var l:Number
            var r:Number
            var p:Number

            this._blocksize=_ba.length / (_resolution * _zoomby)
            shaveblocksize()
            this.graphics.clear()

            this.graphics.beginFill(0xc5c5c5, 0.5)
            this.graphics.drawRect(0, 50, _resolution, 1)

            var max:int=_ba.length - (_blocksize * _resolution)
            var pos:int=Formulas.interpolate(_slider_rel_pos, 0, max)
            _ba.position=pos
            for (var i:int=0; i < _resolution; i++)
            {
                l=Math.abs(_ba.readFloat());
                r=Math.abs(_ba.readFloat());

                l*=50
                r*=50
                p=(l + r)/2

                try
                {
                    this.graphics.drawRect(i, 50, 1, p)
                    this.graphics.drawRect(i, 50, 1, -p)

                }
                catch (e:Error)
                {


                }
                _ba.position+=_blocksize
            }


        }

        private function shaveblocksize():void
        {
            do
            {
                _blocksize--
            } while (_blocksize % 4);
        }

    }
}

Solution Thanks to backtodos for his help. This could do with a hell of a lot of optimization but its good starting point for somebody.

package drawing
{
    import flash.display.Bitmap;
    import flash.display.BitmapData;
    import flash.display.MovieClip;
    import flash.events.MouseEvent;
    import flash.geom.Rectangle;
    import flash.media.Sound;
    import flash.utils.ByteArray;
    import flash.utils.getTimer;

    public class WaveformView extends MovieClip
    {
        private var _sound:Sound
        private var _resolution:Number
        private var _quality:int=1;
        private var _buffersize:int;
        private var _ba:ByteArray=new ByteArray()
        private var _zoomby:int=1;
        private var _zoomXFactor:int=2;
        private var _maximumSecondsPerViewPort:int=10;
        ;
        private var _blocksize:int;
        private var _scrollbar:Bitmap
        private var _scrollthumb:MovieClip


        private var _relativeThumbPosition:Number=0
        private var _sliderRange:int
        private var _sliderCenterMinLeft:int
        private var _sliderCenterMax_Right:int;
        public var _slider_rel_pos:Number=0;
        private var _sliderCenter:Number=0;
        private var _resolutions:Array

        private var _zoomsInSecondsArray:Array=new Array();
        private var _numberOfZoomLevels:int;

        public function WaveformView(snd:Sound, res:Number=800, quality:int=2, buffersize:int=2048, height:int=100)
        {
            super();
            this._sound=snd;
            this._resolution=res;
            this._quality=quality; // not implemented yet
            this._buffersize=buffersize;

            //  addChild(viewportBitmap)
            _resolutions=new Array()

            initScrollBar();
            getPCM()
            calculateZoomLevelData()
            drawwaveform()

        }

        public function calculateZoomLevelData():void
        {
            _zoomsInSecondsArray=[]
            var amt:int=Math.round(_sound.length / 1000);
            _zoomsInSecondsArray.push(amt);

            while (amt >= _maximumSecondsPerViewPort)
            {
                amt=amt / _zoomXFactor;
                if (amt >= _maximumSecondsPerViewPort)
                {
                    _zoomsInSecondsArray.push(amt);
                }
            }

            _numberOfZoomLevels=_zoomsInSecondsArray.length;


            var checkSize:int=_resolution;
            var r:ResolutionCache
            r=new ResolutionCache(checkSize)

            _resolutions.push(r)



            for (var c:int=1; c < _numberOfZoomLevels + 1; c++)
            {
                checkSize=checkSize * _zoomXFactor;
                r=new ResolutionCache(checkSize)
                _resolutions.push(r)
            }
            _resolutions.pop()

        }

        private function initScrollBar():void
        {
            var sbbmd:BitmapData=new BitmapData(_resolution, 20, false, 0xcccccc)
            _scrollbar=new Bitmap(sbbmd)
            _scrollbar.y=120
            addChild(_scrollbar)

            sbbmd=new BitmapData(_resolution, 16, false, 0xeeeeee)
            _scrollthumb=new MovieClip()
            _scrollthumb.graphics.beginFill(0xff0000)
            _scrollthumb.graphics.drawRect(0, 0, _resolution, 10)
            _scrollthumb.graphics.endFill()
            _scrollthumb.y=125
            addChild(_scrollthumb)
            _scrollthumb.buttonMode=true
            _scrollthumb.addEventListener(MouseEvent.MOUSE_DOWN, beginthumbdrag)

        }

        private function beginthumbdrag(e:MouseEvent):void
        {
            _scrollthumb.startDrag(false, new Rectangle(_scrollbar.x, _scrollbar.y + 5, _scrollbar.width - (_scrollthumb.width), 0))
            _scrollthumb.addEventListener(MouseEvent.MOUSE_MOVE, shuttleMoving, false, 0, true);
            _scrollthumb.addEventListener(MouseEvent.MOUSE_UP, endthumbdrag, false, 0, true);
        }

        private function endthumbdrag(e:MouseEvent):void
        {
            _scrollthumb.stopDrag();
            e.updateAfterEvent();
            _scrollthumb.removeEventListener(MouseEvent.MOUSE_MOVE, shuttleMoving);
        }

        private function shuttleMoving(e:MouseEvent):void
        {
            calculateShuttleRelativePosition();
            drawwaveform()
            //e.updateAfterEvent()
        }

        private function calculateShuttleRelativePosition():void
        {

            var _x:Number=_scrollthumb.x
            _sliderCenter=_x + (_scrollthumb.width / 2)
            _sliderRange=_scrollbar.width - _scrollthumb.width;
            _slider_rel_pos=(_sliderCenter - _sliderCenterMinLeft) / _sliderRange

        }

        public function getPCM():void
        {
            var len:int=_sound.length * 44.1
            _sound.extract(_ba, len)
            this._blocksize=_ba.length / (_resolution * _zoomby)
            shaveblocksize()
        }

        public function zoomin():void
        {
            if (this._zoomby < 16)
            {
                this._zoomby*=2
                resizeThumb()
                drawwaveform()
            }
        }

        public function zoomout():void
        {
            if (_zoomby >= 2)
            {
                this._zoomby/=2
                resizeThumb()

                drawwaveform()
            }
        }

        private function resizeThumb():void
        {
            _scrollthumb.width=_scrollbar.width / _zoomby
            if (_scrollthumb.width == _resolution)
            {
                _slider_rel_pos=0
            }
            _sliderCenterMinLeft=_scrollthumb.width / 2;
            _sliderCenterMax_Right=_scrollbar.width - (_scrollthumb.width / 2);
            _sliderRange=_scrollbar.width - _scrollthumb.width;
            _sliderRange=_scrollbar.width - _scrollthumb.width;
            _scrollthumb.x=(_slider_rel_pos * _sliderRange)
        }

        private function getResolutionCache(vww:int):ResolutionCache
        {
            var r:ResolutionCache
            for (var i:int=0; i < _resolutions.length; ++i)
            {
                if (_resolutions[i].name == vww)
                {
                    r=_resolutions[i];
                    break;
                }
            }
            return r;

        }

        public function drawwaveform():void
        {
            var starttime:Number=getTimer()
            var readposition:int
            var viewPortWidth:int=_resolution;
            var virtualWindowWidth:int=_resolution * _zoomby;
            var l:Number
            var r:Number
            var p:Number

            var rc:ResolutionCache=getResolutionCache(virtualWindowWidth)

            this._blocksize=_ba.length / virtualWindowWidth
            shaveblocksize()
            this.graphics.clear()

            this.graphics.beginFill(0xc5c5c5, 0.5)
            this.graphics.drawRect(0, 50, _resolution, 1)

            var pixel:int=(_slider_rel_pos * (virtualWindowWidth - viewPortWidth - 1));
            var readpoint:int=pixel * _blocksize;
            var rect:Rectangle
            _ba.position=readpoint;

            this.graphics.clear()
            this.graphics.beginFill(0xffffff)

            for (var i:int=0; i < _resolution; i++)
            {

                if (rc.points[pixel] == -1)
                {
                    _ba.position=pixel * _blocksize
                    l=_ba.readFloat() * 100;
                    r=_ba.readFloat() * 100;

                    p=(l + r) / 2

                    rc.addPoint(pixel, i, height / 2, p)
                    rect=rc.points[pixel]
                    //this.graphics.drawRect(rect.x,rect.y,rect.width,rect.height)
                    this.graphics.drawRect(i, 50, 1, p)
                    this.graphics.drawRect(i, 50, 1, -p)


                }
                else
                {

                    //viewportBitmapdata.fillRect(rc.points[pixel], 0xffffff)
                    rect=rc.points[pixel]
                    this.graphics.drawRect(i, 50, 1, rect.height)
                    this.graphics.drawRect(i, 50, 1, -rect.height)
                        //this.graphics.drawRect(i,height/2,rect.width,rect.height)


                }

                pixel++

            }

        }

        private function shaveblocksize():void
        {
            do
            {
                _blocksize--
            } while (_blocksize % 4);
        }

    }
}

Answer 1

Why exactly is it, that you try to find maximum over some chunks?

Also, what is this code supposed to do:

            if (_pcmData.bytesAvailable)
            {
                var la:Number=_pcmData.readFloat();
                var ra:Number=_pcmData.readFloat();
                var l:Number=la > 0.0 ? la : -la;
                var r:Number=ra > 0.0 ? ra : -ra;

                ++_divCount;
                var ml:Number=0;
                var mr:Number=0;
                var a_ml:Number=ml > 0.0 ? ml : -ml;
                var a_mr:Number=mr > 0.0 ? mr : -mr;

                ml=a_ml > (l) ? ml : l;
                mr=a_mr > (r) ? mr : r;
            }

ml and mr are always 0, so what is the point of checking that? and what is divCount?

If it is for drawing a waveform, I'd draw it with a certain resolution. If I want to draw a waveform on an area 500 pixels wide, than basically 500 samples will do the job. If you want it to average out a little, than a hand full of samples per pixel really should be enough. So from the time frame represented by a pixel, you choose 4 samples (equidistant or random) and calculate the average. Should the variance exceed a certain treshold, you could try to get a more reliable value by getting more samples. In general, waveforms are continuous and smooth. The time to draw the graph should scale linearly with the size of the graph, not with the number of samples involved.

greetz
back2dos