1 (edited by paulkinzelman 2010-09-07 18:11:42)

Topic: Can the tool import multiple point-series curves?

I didn't see a whole lot about the format of the import files for a point series in the documentation, thus this question...

Is there a way to import several curves-worth of series of points from one file?

In other words, I want to have one series of points describe one curve, and then a second series of points describe a second curve, etc., so I will end up with a family of curves all in the same graph. Is there a way to embed commands or something in the import file where I can put all the points for all the curves in the same file? Or do I have to import each curve's-worth of points separately?

In other words, kind of a scripting input like:
Curve: Test1
Color: Green
Trendline: Linear
Points:
1;2
3;4
Curve: Test2
Color: Blue
Trendline: Linear
Points:
5;6
7;8
9;10
[etc.]

TIA! Nice little tool!

Re: Can the tool import multiple point-series curves?

paulkinzelman wrote:

I didn't see a whole lot about the format of the import files for a point series in the documentation, thus this question...

There is a little in the help file, but you are probably right that I need to improve the documentation of import of point series.

paulkinzelman wrote:

Is there a way to import several curves-worth of series of points from one file?

Yes, but only if they use the same x-coordinates.

paulkinzelman wrote:

In other words, I want to have one series of points describe one curve, and then a second series of points describe a second curve, etc., so I will end up with a family of curves all in the same graph. Is there a way to embed commands or something in the import file where I can put all the points for all the curves in the same file? Or do I have to import each curve's-worth of points separately?

When the data points do not share the same x-coordinates, you have to put each point series in a separate file. The format could look like this:

1;2
3;4
5;6
7;8

Graph supports separation by semicolon, comma, tabulator or space.

The beta version of Graph 4.4 has support for scripting, but it has even less documentation than the import feature. smile

Re: Can the tool import multiple point-series curves?

OK, how about this deal... I'll install the beta, and if you give me a few hints on how to get it going, I'll write up a section about how to use scripting for the documentation if I can get it to work. I've got a good background in computers - ~30 years in engineering. Might be easier to respond privately to paul@kinzelman.com

However, I do need to not necessarily have the same X coordinates for all points, and I don't understand the reason for the restriction if all the points specify both x and y. Seems a small change to go from allowing different files to specify different X coordinates to munging them all together into one file with some kind of 'EOF' marker or something. It's a lot easier to run emacs macros on one big honkin' file than a bazillion separate ones.

Also, as a suggestion, I'm also fiddling with the codelite.org tool and their documentation is in the wiki form so that users can update sections on their own. I've already updated one thing that wasn't documented and was having trouble with. I think that's a great way to get the community to help with the documentation so you don't have to do it yourself.

Re: Can the tool import multiple point-series curves?

paulkinzelman wrote:

OK, how about this deal... I'll install the beta, and if you give me a few hints on how to get it going, I'll write up a section about how to use scripting for the documentation if I can get it to work. I've got a good background in computers - ~30 years in engineering. Might be easier to respond privately to paul@kinzelman.com

You got yourself a deal. smile
But I think it is best to keep responding in the forum as it might have interest to others.

You need to install Python 3.1 to use the plugin scripting feature. You can access it by pressing F11 inside Graph, which will show a Python interpreter console, or you can place a .py file in the Plugins directory, which will be automatically run when Graph is started.

There are already a few example plugins installed with Graph you can look at.

paulkinzelman wrote:

However, I do need to not necessarily have the same X coordinates for all points, and I don't understand the reason for the restriction if all the points specify both x and y. Seems a small change to go from allowing different files to specify different X coordinates to munging them all together into one file with some kind of 'EOF' marker or something. It's a lot easier to run emacs macros on one big honkin' file than a bazillion separate ones.

It is probably mostly because no one has asked about it before. I think csv files are mostly used to import data points from Excel or a similar program, where you have the x-coordinates in the first column and the y-coordinates for every point series in the other columns.

paulkinzelman wrote:

Also, as a suggestion, I'm also fiddling with the codelite.org tool and their documentation is in the wiki form so that users can update sections on their own. I've already updated one thing that wasn't documented and was having trouble with. I think that's a great way to get the community to help with the documentation so you don't have to do it yourself.

I will consider using a wiki, but I don't think it will work very well with Graph. The users of Graph are very different from the users of CodeLite.

Re: Can the tool import multiple point-series curves?

>You need to install Python 3.1 to use the plugin scripting feature.

I don't know anything about Python, I'll have to install it and see what it is. If it's scripting and there are examples, I can probably stumble thru it, I've used Perl, Expect, etc., but it's been loooooooog while.

Re: multiple curves in an import

Would it be possible for you to toss something new in the beta and we can try to get it to work while I'm stumbling into Python? It'd be *very* convenient for what I want to do. I'd rather spend time with you getting the single file thing to work than time creating a bunch of separate files, because I bet if the feature is there, others will find it very useful too. And it doesn't sound like that big of a change if you've already got a lot of scripting code in there.

Re: wiki
>The users of Graph are very different from the users of CodeLite.

Would you be politely saying that I'm a multiple-personality schizophrenic? :-) :-)

Re: Can the tool import multiple point-series curves?

I don't know anything about Python, I'll have to install it and see what it is. If it's scripting and there are examples, I can probably stumble thru it, I've used Perl, Expect, etc., but it's been loooooooog while.

Python is a scripting language. When you know other programming languages it should be easy to learn. There is a great tutorial in the documentation.


Would it be possible for you to toss something new in the beta and we can try to get it to work while I'm stumbling into Python? It'd be *very* convenient for what I want to do. I'd rather spend time with you getting the single file thing to work than time creating a bunch of separate files, because I bet if the feature is there, others will find it very useful too. And it doesn't sound like that big of a change if you've already got a lot of scripting code in there.

If it is just a matter of adding some kind of separator, I guess it would be easy. I will take a look at it. Graph itself is written in C++, so there isn't any scripting code in it yet though.

Would you be politely saying that I'm a multiple-personality schizophrenic? :-) :-)

Maybe I am. smile
At least I don't think most users of Graph are programmers and are not used to writing documentation.

Re: Can the tool import multiple point-series curves?

Re: Python
I'm sure I can, it's just a matter of time, I'll get to it.

One thing I don't understand is the line between what Python does and what Graph does, but maybe it'd be more clear if/when I get to Python. Do I need the Graph beta to run Python? I tried F11, but nothing happened. I'm running Graph 4.3 build 384. This is quite a diversion from what I was originally trying to do tho. Good thing I'm not in a hurry. :-)

If you could put whatever I need to do multiple point-sequence curves into the beta and ship it to me with a few hints, I'll play with it. Then I'll have my coding personality talk with my documentation personality and see what he can come up with. :-)

Actually, I like writing stuff up, first of all for me so when I forget, I can go back and remember it. And when I explain something (or write it down), it helps me learn it well.

Re: Can the tool import multiple point-series curves?

OK, I've installed "the full Monty" ... Python that is :-) and skimmed the docs, and I still don't see how it connects to Graph. If a Python script is able to do Graph commands, then I don't see why I can't have a Python script do multiple point-curves and points with different X values, and in fact, just about anything that Graph can do without help from Graph.

Perhaps it'd help if you knew my application...

I've got some performance curves in graphical form created from experimentation. I want to write software that analyzes/uses these graphs, so I've created a set of points that I manually read from the graph, which I'm hoping is a piece-wise linear approximation to the curves.

And I'd like to make sure that the point data describing the curves that I've put into C initialize statements looks about right, so I'd like to process the C initializer data into a set of points that I can display on your Graph program and look for inaccuracies. For instance, maybe I read a graph point wrong and a point is misplaced. I can't tell that from just looking at numbers. I need to see the graph of it.

When I find an error or an unrealistic bend, I'd fix it in the C initializers, then run it thru again and replot it in Graph to make sure the result is smooth and accurate. So as you can see, I'm expecting multiple passes.

And I'll probably have to write something to munge the C initializer format into whatever Graph needs. One graph I've got has about 18 different curves, so doing 18 individual files would be inconvenient to do once, nevermind doing the set multiple times.

Does that make sense?

Seems like anybody doing software analysis of experimental data might be interested in this process. I can post my software format munger later if I do one in case anybody is interested.

Re: Can the tool import multiple point-series curves?

paulkinzelman wrote:

One thing I don't understand is the line between what Python does and what Graph does, but maybe it'd be more clear if/when I get to Python. Do I need the Graph beta to run Python? I tried F11, but nothing happened. I'm running Graph 4.3 build 384. This is quite a diversion from what I was originally trying to do tho. Good thing I'm not in a hurry. :-)

You need the beta version of Graph 4.4 to use the plugin system. The plugin system is just a way to use Python scripts to interact with Graph.

paulkinzelman wrote:

If you could put whatever I need to do multiple point-sequence curves into the beta and ship it to me with a few hints, I'll play with it.

If you download the latest beta version (build 483) you can start a line with a # in a csv file to indicate a new point series. For example if you import a file containing this, you will get two point series:

1;2
3;4
5;6
#
3;1
5;2
8;-4
paulkinzelman wrote:

OK, I've installed "the full Monty" ... Python that is :-) and skimmed the docs, and I still don't see how it connects to Graph. If a Python script is able to do Graph commands, then I don't see why I can't have a Python script do multiple point-curves and points with different X values, and in fact, just about anything that Graph can do without help from Graph.

You could of course write a whole graph plotting application in Python, but I think that would be a lot of work. The idea is that you can write extensions to Graph. For example there is an example plugin installed with Graph that allows you to create circles by entering radius and center coordinate. It actually just creates a parametric function. The other example plugin will show some statistics information for a point series.

paulkinzelman wrote:

And I'd like to make sure that the point data describing the curves that I've put into C initialize statements looks about right, so I'd like to process the C initializer data into a set of points that I can display on your Graph program and look for inaccuracies. For instance, maybe I read a graph point wrong and a point is misplaced. I can't tell that from just looking at numbers. I need to see the graph of it.

Thanks for the explanation. I think I understand what you are doing. If you download the latest beta version you should be able to import several point series from one file by using the format shown above without having to write any Python code.

Re: Can the tool import multiple point-series curves?

OK, now we're getting somewhere. I understand better about Python, and I think you're right, it really doesn't add any value to what I'm doing. But the "#" thing does.

I think you've implemented the bare minimum, but while I've got your ear, let me make a suggestion that would make life (mine anyway :-) much easier.

Suppose you were to ignore any characters that you didn't understand in an import file and look only for numbers - specifically pairs of numbers on a line, and the # symbol. Then I could feed my .H file in directly if I were careful about my comments. The file looks like this:
/*****************************************************************
*Here's a bunch of comments, some with a single
*number in the line like copyright 2010.
*/

#include "ga.h"

point_t
tbl_lr35_to8_wt[MX_CURVES][MX_POINTS] =
{
  {//Curve 0
    {NULLNR, NULLNR},
    {10000, 1980},
    {12400, 2150},
  },
//# (for Graph to know it's another curve)
  {//Curve 1
    {NULLNR, NULLNR},
    {10000,  2130},
    {12400,  2310},
    {12600,  2330},
  }
}

So if you take the browser philosophy of being as error tolerant as possible, I could import this file directly into Graph and it'd just ignore the lines it can't understand, and plot only the point lines with two numbers. Or even better, it'd be neat if I could embed commands in the C-comments for Graph like:
//Color: Blue
//Line: Interpolation Linear
etc.

What do you think?

Re: Can the tool import multiple point-series curves?

I just tried your beta and with a small amount of emacs munging, I can get my data into the right format. And in fact, I already found a couple of typos in my data points!

It'd be nice to be able to turn off the big markers that signify the actual points, as they cover a lot of area. I can select the type, but they're all big. It'd be nice to select either off or very tiny, like 2x the width of the line.

Re: Can the tool import multiple point-series curves?

paulkinzelman wrote:

Suppose you were to ignore any characters that you didn't understand in an import file and look only for numbers - specifically pairs of numbers on a line, and the # symbol. Then I could feed my .H file in directly if I were careful about my comments. The file looks like this:

It is actually not as easy as it might sound. Graph handles expressions and not only numbers, so you can actually have a coordinate like (pi+2,e^-1)

paulkinzelman wrote:

So if you take the browser philosophy of being as error tolerant as possible, I could import this file directly into Graph and it'd just ignore the lines it can't understand, and plot only the point lines with two numbers. Or even better, it'd be neat if I could embed commands in the C-comments for Graph like:
//Color: Blue
//Line: Interpolation Linear
etc.

Now you are starting to invent a new scripting language, and I already have Python for that. smile


paulkinzelman wrote:

I just tried your beta and with a small amount of emacs munging, I can get my data into the right format. And in fact, I already found a couple of typos in my data points!

Great!

paulkinzelman wrote:

It'd be nice to be able to turn off the big markers that signify the actual points, as they cover a lot of area. I can select the type, but they're all big. It'd be nice to select either off or very tiny, like 2x the width of the line.

Under the type you can select the size. You can even set it to 0 if you don't want the markers shown.

Re: Can the tool import multiple point-series curves?

Re: ignore characters

I see now the problem with my suggestion. It's wonderful how easy things are when suggestions are made from a place of ignorance! :-)

Re: scripting language

I don't understand yet how Python connects to Graph and I don't see it in any of the menu items. How about a short example to do something trivial?

Re: size

I see now how to get rid of the markers, that'll do what I want, but I have to do it for each trace one by one. It'd be neat if I could do multiple or all the traces at one time. Or is that what Python scripting is for? Maybe you could just make an example of how to do that.

14 (edited by paulkinzelman 2010-09-11 18:52:21)

Re: Can the tool import multiple point-series curves?

It looks like when an import has a blank line (perhaps in conjunction with the new "#" command you put in), the program crashes.

Also, if my first value is "0,1540" nothing displays and it got imported as X=0,1540 and Y is blank.
I had to put a space in: "0, 1540"

Re: Can the tool import multiple point-series curves?

Unfortunately I am not able to reproduce the two problems you describe. Can you maybe attach a file that demonstrates the problems?

Re: Can the tool import multiple point-series curves?

I'm unable to reproduce the crash now (I tried a bunch of stuff on what was pretty close to the file as it was when the import crashed I think. Oh well...)

But if you import the following file:
0,1540
10800,123
15000,150

It thinks both numbers on each line are X, and there's nothing for Y.
But it works if you insert a space after each comma like:
0, 1540
10800, 123
15000, 150

BTW, it'd be neat if you had a 'clear' or 'close' command so I wouldn't have to exit and start the program again for each attempt to reproduce the crash.

Re: Can the tool import multiple point-series curves?

I am still not able to reproduce the problem. It seems to work fine no matter what I try. Could you please attach a file that fails to import. There must be something else to it.

I am not sure what you mean with 'clean' or 'close' command. You can use File|New to create a new coordinate system.

18 (edited by paulkinzelman 2010-09-11 23:45:07)

Re: Can the tool import multiple point-series curves?

Oh, yes, looks like New will do what I want (create a new system), thanks!

I'm using V4.4 Beta, build 483.

OK, here's the file. When I import, then edit the curve, I see:
     X             |      Y
0,1540          |
10800,123    |
15000,150    |

and nothing plots.

Post's attachments

Attachment icon tst.txt 31 b, 714 downloads since 2010-09-11 

Re: Can the tool import multiple point-series curves?

The reason is that there is a space at the end of the first line. This seems to confuse Graph. I will look into that. But it seems to work if you remove the space.

Re: Can the tool import multiple point-series curves?

Interesting - I got it to work by putting a space after each comma even with the extra space at the end of the first line. And it's strange that a space after the first line messes up the interpretation of the subsequent lines as well. I had to put a space after all the commas.

Re: Can the tool import multiple point-series curves?

The reason is that Graph tries to guess what the separator is. When it finds a space in the first line it will use space as the separator and look for a space in every line. When there is no space, it will instead use comma as separator. I will change it so it ignores spaces in front and end of the line.

22 (edited by paulkinzelman 2010-09-18 01:05:57)

Re: Can the tool import multiple point-series curves?

OK, I reproduced the crash on the import of the multiple curves, probably the one I saw before but couldn't reproduce. It turns out if I have a line composed *only* of spaces, I get an exception at EEFFACE

Note that line 3 has a couple of spaces and nothing else.
A blank line doesn't crash.

It must be allergic to spaces. :-)

Post's attachments

Attachment icon tst.txt 37 b, 708 downloads since 2010-09-18 

Re: Can the tool import multiple point-series curves?

Yes, it looks like Graph had an allergy for spaces. I have fixed it now in the beta version.

Re: Can the tool import multiple point-series curves?

Just tried it - looks like you successfully inoculated it against spaces. :-)

Re: Can the tool import multiple point-series curves?

Using Point Import Wizard for Alibre Design, you can easily import large sets of points as points, connected lines, or as a b-spline of configurable polynomial order on to either an existing sketch or a new sketch based on a design plane.