Charting Limits?

A forum dedicated to WinForms version of LightningChart Ultimate.

Moderator: Arction_LasseP

plotter
Posts: 41
Joined: Thu Apr 06, 2017 2:29 am

Charting Limits?

Post by plotter » Tue Oct 17, 2017 8:25 pm

Hello Arction Support,

I have implemented the WinForm Charting component and seem to have hit some limits with my data.

It seems that the visible point count is the limiting factor in the product, is that correct?
For instance, when I take the zoom bar chart example and change the point count to 50,000,000, I get out of memory errors.
When I lower it to 30,000,000, It does eventually display the main chart, but the top chart takes a very long time to display (maybe due to the AreaSeries instead of PointLineSeries?). When it does display, there is a huge lag when trying to navigate a large amount of visible points. So, is the visible point count the effective limiting factor in the charts ?

Most of my charts have dozens of Y Axes and Series over several segments, with very good performance up to about 500,000 visible points total. Above that, the interface gets quite sluggish when panning and zooming large amounts of visible points.
Even with large point counts in the series, the navigation performance seems really good, until the visible point count gets really high.

It seems that above certain thresholds, out of memory conditions occur.

Presumably during development, you have tested many various point count scenarios, and it would be great to know just where the limits actually are...

I have large datasets to visualize (22,000,000,000 points and growing, and have a few charts that now have 30,000,000 points /chart (distributed over 20+ series ) that are getting to be unusably slow to navigate, as is... (the charts with < 3,000,000 visible point counts work fine)

So, for these higher point count charts, is the solution to summarize or bin the points to create a lower density series when needing to display large x ranges? Or is there another technique that is easier/better?

I have points that update once a second and the customer wants to see a years worth at a time...

Thanks,
Heather

ArctionKestutis
Posts: 386
Joined: Mon Mar 14, 2016 9:22 am

Re: Charting Limits?

Post by ArctionKestutis » Wed Oct 18, 2017 11:06 am

Hello Heather,

Arction don't have the official point count limit. Unfortunately, .Net Array has - that is, no single object can be larger than 2GB.

It seems that in 'Zoom Bar Chart' test you are using AreaSeries. This series (same as HighLowSeries) is heavier than other series. The Draw-data contains at least 48 bytes per point. Therefore, the required array size is hitting 2 GB limit. If you want to display high number of points use simpler series: e.g. PointLineSeries or SampleDataSeries (they had lower memory requirement).

Even if you find way to limit Point array data to 2GB, the performance still going to depend on capabilities of your system. LightningChart is not a limiter here. This is there your CPU, GPU and memory counts. With programs like ‘Process Explorer’ (https://technet.microsoft.com/en-us/sys ... lorer.aspx ) you could see how your app is using machine resources. This should help you to identify limiting factors on your environment.

Following tips will help you to optimize any chart not only with high point count. First, try to avoid creating unnecessary draw data. For example, don't draw boarders, if you don't need them; don't draw points, but only lines; use LineStyle.Width=1. Second, minimize mouse interaction and movement of series (zooming/panning). Those things is the reason data recalculated and redrawn. Enabled ViewXY.DropOldSeriesData property, which will destroy out-scrolled data and save memory.

If you still want to plot all the tens of millions points, I would suggest to use one static chart (just to plot all the points) and second chart for interaction (limited range, zoomed-in).
If those recommendations not suitable for you, when you need to come-up with some down-sampling technique and algorithm when to use it.

Hope this helps.
All the best,
Kestutis

plotter
Posts: 41
Joined: Thu Apr 06, 2017 2:29 am

Re: Charting Limits?

Post by plotter » Wed Oct 18, 2017 4:11 pm

Thank you Kestutis,

You have given me some new ideas.
I an testing on a 128GB ~5Ghz Workstation with 2 GTX 1080Ti's, so I don't think machine hardware resources are the issue.

Yes, I see how area series can be a heavier load.

So if the performance tuning I need is accomplish is based on reducing the rendering load of the visible objects,
how do mouse and other chart events affect rendering speed?

I am using mouse over events for custom highlighting, does this affect performance during panning/zooming even withing triggering the event?

I am using range changed and position changed events to drive views, it seems that there is no setting to determine the granularity of the value change that triggers the event, would changing the strategy to use a timer based manual evaluation of the multitouch/mouse events help as well?

Or is there a way to control the refresh rate of the range/value changed events?

Also, is there a hardware rendering engine of choice for best performance?

And, what effect does anti aliasing have? and what is the most performance enhancing setting?
I noticed various other rendering properties, are there any other specific settings that may help?

If I enable ViewXY.DropOldSeriesData, what does this affect when the user wants to see this data again? Do I need to reload it?
I really would to avoid trips to the database, if possible...

Are there particularly 'costly' pan / zoom settings to avoid/minimize their use? (like only pan or zoom one direction at a time (horizontal only/vertical only) or disable rectangle zooming?

I think I will be implementing some background downsampling based on initial object point counts for the 'full data' view chart.

Am I penalized for creating (multiple) series and not displaying them? (As long as I have available memory to do so)
I am thinking if I initially create and load a downsampled series for initial 'big picture' view, and in the background load the full actual data, can I get a x value of the downsampled series, then calculate the Y value from the 'hidden' full series, in order to present the specific subset of data with regular sampling?

Or does a series need to be displayed in order to be evaluated?

Also I read somthing about displaying points with rectangle solid shapes vs other shapes or transparent? how does this affect performance?

Sorry to need a lot of details here, but I hope it will help in implementing Lightning Charts in the most effective manner for my application.

I want to thank you for all your help to date.
You have made it possible for me to develop a truly first class product using Lightning Charts.

Heather
Last edited by plotter on Thu Oct 19, 2017 11:13 pm, edited 1 time in total.

User avatar
ArctionPasi
Posts: 1366
Joined: Tue Mar 26, 2013 10:57 pm
Location: Finland

Re: Charting Limits?

Post by ArctionPasi » Thu Oct 19, 2017 1:20 pm

Hi Heather,

please check that "Prefer 32-bit" is not enabled in your Project options. That may prevent using more than 2GB memory
LightningChart Support Team, PT

ArctionKestutis
Posts: 386
Joined: Mon Mar 14, 2016 9:22 am

Re: Charting Limits?

Post by ArctionKestutis » Thu Oct 19, 2017 2:44 pm

Hello Heather,

The amount of questions you send is not for a single forum topic, but requires whole user's manual :D

I will send you replies to those question by email. If other users found those question interesting please write to Arction's support account and open new forum topic.

All the best,
Kestutis

plotter
Posts: 41
Joined: Thu Apr 06, 2017 2:29 am

Re: Charting Limits?

Post by plotter » Thu Oct 19, 2017 11:13 pm

Thanks again Kestutis,

I appreciate all the efforts from you and your team.
I have no regrets choosing Lightning Charts to integrate into my products.

Heather