1

I have 20 milliion rows in my table.I am trying to plot graph for timestamp using http://www.jqplot.com/tests/date-axes.php (date vs time) how will jquery handle so much of data to plot it. Two questions:

1.How can I construct an array from the below code:

line1=[['2008-09-30',4], ['2008-10-30',6], ['2008-11-30',5], ['2008-12-30',9], ['2009-01-30',8]];`

2.How can I improvize the code in this case:

def get_data(request):
 cdr = Cdr.objects.values_list('start_of_call',flat=True)
 context_instance=RequestContext(request, {})) 
 return render_to_response('cdr/get_data.html', context_instance=RequestContext(request, {'cdr':cdr}))

template:

<script class="code" type="text/javascript">
$(document).ready(function(){

{% for c in cdr %}
  {{ c|date:"d M Y"}}
{% endfor %}
 How to construct the below array line from for loop or using jquery methods
//var line1=[['2008-09-30',4], ['2008-10-30',6], ['2008-11-30',5], ['2008-12-30',9], ['2009-01-30',8]];

var plot1 = $.jqplot('chart1', [line1], {
title:'Default Date Axis',
axes:{xaxis:{renderer:$.jqplot.DateAxisRenderer}},
series:[{lineWidth:4, markerOptions:{style:'square'}}]
});
});
 </script>

EDIT

  line1=[['2008-09-30',4], ['2008-10-30',6], ['2008-11-30',5], ['2008-12-30',9], ['2009-01-30',8]];`

In the above line 4,6,5,9,8 are the hours

EDIT2

CDR values:

    [datetime.datetime(2012, 9, 13, 15, 54, 52, tzinfo=<UTC>), datetime.datetime(2012, 9, 22, 18, 24, 18, tzinfo=<UTC>), datetime.datetime(2012, 9, 6, 7, 42, 50, tzinfo=<UTC>), datetime.datetime(2012, 9, 7, 21, 27, 13, tzinfo=<UTC>), datetime.datetime(2012, 9, 3, 11, 57, 9, tzinfo=<UTC>), datetime.datetime(2012, 9, 1, 17, 23, 41, tzinfo=<UTC>), datetime.datetime(2012, 9, 15, 6, 14, 33, tzinfo=<UTC>), datetime.datetime(2012, 9, 4, 5, 36, 30, tzinfo=<UTC>), datetime.datetime(2012, 9, 17, 10, 45, 6, tzinfo=<UTC>), datetime.datetime(2012, 10, 1, 3, 1, 17, tzinfo=<UTC>), datetime.datetime(2012, 9, 27, 8, 36, 41, tzinfo=<UTC>), datetime.datetime(2012, 9, 30, 3, 12, 18, tzinfo=<UTC>), datetime.datetime(2012, 9, 18, 4, 8, 49, tzinfo=<UTC>), datetime.datetime(2012, 9, 2, 7, 21, tzinfo=<UTC>), datetime.datetime(2012, 9, 19, 16, 57, 34, tzinfo=<UTC>), datetime.datetime(2012, 9, 17, 17, 14, 49, tzinfo=<UTC>), datetime.datetime(2012, 9, 1, 22, 40, 33, tzinfo=<UTC>), datetime.datetime(2012, 9, 15, 1, 7, 39, tzinfo=<UTC>), datetime.datetime(2012, 9, 22, 17, 11, 45, tzinfo=<UTC>), datetime.datetime(2012, 9, 10, 3, 16, 21, tzinfo=<UTC>), '...(remaining elements truncated)...']
9
  • can you post the value of cdr Commented Nov 27, 2012 at 11:15
  • I'm not altogether clear on what you are trying to do. Can you give an example of the values in cdr (and maybe give it a better name). you are trying to build the string line1=[['2008-09-30',4], ['2008-10-30',6],... in your template, right? Why not use JSON to translate Python data into the corresponding JavaScript structure? Commented Nov 27, 2012 at 11:16
  • cdr values are 2012-09-28 14:37:22+00:00 ,2012-09-21 14:37:22+00:00,....... Commented Nov 27, 2012 at 11:23
  • @scytale:I am trying to plot a graph using the given values Commented Nov 27, 2012 at 11:25
  • can you show cdr as a python pretty-printed data structure? and any reason not to use JSON? Commented Nov 27, 2012 at 11:33

1 Answer 1

3

something like this:

from django.utils import simplejson

def get_data(request):
    cdr = [(x.strftime('%Y-%M-%D'), x.hour) for x in \
            Cdr.objects.values_list('start_of_call',flat=True)]
    return render_to_response('cdr/get_data.html',
                              {'cdr': simplejson.dumps(cdr)},
                              context_instance=RequestContext(request))

The format string in strftime might be a bit off - check it here: http://docs.python.org/2.7/library/datetime.html#strftime-strptime-behavior

by the way your view did some weird things with RequestContext - the above is the more usual style.

template:

line1 = jQuery.parseJSON({{ cdr }})

As for issues with sample size - there's no clear answer - it's going to depend on your network connection, your browser and machine. That siad 20M is very big - consider sampling the dataset to bring it down to a more manageable size - experimentation is going to be the best way.

Sign up to request clarification or add additional context in comments.

2 Comments

I really didnt want to go this way(using list comprehension).As i have illustrated the problem correctly.I want to really find an optimal solution for the millions of rows.+1 for all the help.Appreciate it
well you're going to have to iterate over cdr at some stage to convert the datetime objects into (str, int). Anyway you should ask 1 question at a time. Your question didn't ask about an optimal solution - you asked how to construct a JS array from your Python datastructure and secondly how to "improvize" some code, which I understood to mean how to alter your view to pass that datastructure. if you want to know how to optimise the code then you should ask a new question.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.