Storing Python dictionary data into a csv

I have a list of dicts that stores Facebook status data (Graph API):

len(test_statuses)
3


test_statuses
[{u'comments': {u'data': [{u'created_time': u'2016-01-27T10:47:30+0000',
 u'from': {u'id': u'1755814687982070', u'name': u'Fadi Cool Panther'},
 u'id': u'447173898813933_447182555479734',
 u'message': u'Sidra Abrar'}],
 u'paging': {u'cursors': {u'after': u'WTI5dGJXVnVkRjlqZFhKemIzSTZORFEzTVRneU5UVTFORGM1TnpNME9qRTBOVE00T1RFMk5UQT0=',
 u'before': u'WTI5dGJXVnVkRjlqZFhKemIzSTZORFEzTVRneU5UVTFORGM1TnpNME9qRTBOVE00T1RFMk5UQT0='}},
u'summary': {u'can_comment': False,
u'order': u'ranked',
u'total_count': 1}},
u'created_time': u'2016-01-27T10:16:56+0000',
u'id': u'5842136044_10153381090881045',
u'likes': {u'data': [{u'id': u'729038357232696'},
{u'id': u'547422955417520'},
{u'id': u'422351987958296'},
{u'id': u'536057309903473'},
{u'id': u'206846772999449'},
{u'id': u'1671329739783719'},
{u'id': u'991398107599340'},
{u'id': u'208751836138231'},
{u'id': u'491047841097510'},
{u'id': u'664580270350825'}],
u'paging': {u'cursors': {u'after': u'NjY0NTgwMjcwMzUwODI1',
 u'before': u'NzI5MDM4MzU3MjMyNjk2'},
u'next': u'https://graph.facebook.com/v2.5/5842136044_10153381090881045/likes?limit=10&summary=true&access_token=521971961312518|121ca7ef750debf4c51d1388cf25ead4&after=NjY0NTgwMjcwMzUwODI1'},
u'summary': {u'can_like': False, u'has_liked': False, u'total_count': 13}},
u'link': u'https://www.facebook.com/ukbhangrasongs/videos/447173898813933/',
u'message': u'Track : Ik Waar ( Official Music Video )\nSinger : Falak shabir ft DJ Shadow\nMusic by Dj Shadow\nFor more : UK Bhangra Songs',
u'shares': {u'count': 7},
u'type': u'video'},
{u'comments': {u'data': [],
u'summary': {u'can_comment': False,
u'order': u'chronological',
u'total_count': 0}},
u'created_time': u'2016-01-27T06:15:40+0000',
u'id': u'5842136044_10153380831261045',
u'likes': {u'data': [],
u'summary': {u'can_like': False, u'has_liked': False, u'total_count': 0}},
u'message': u'I want to work with you. tracks for flicks',
u'type': u'status'}]

I need to extract each status text and the text of each comment under the status, which I can do by appending them to separate lists e.g.,:

status_text = []
comment_text = []

for s in test_statuses:
try:
    status_text.append(s['message'])
    for c in s['comments']['data']:
        comment_text.append(c['message'])
except:
    continue

This gives me two lists of separate lengths len(status_text) = 2, len(comment_text) = 49.

Unfortunately that's a horrible way of dealing with the data since I cannot keep track of what comment belongs to what status. Ideally I would like to store this as a tree structure and export in into a cvs file, but I can't figure out how to do it.

Probable data acv data structure:

    Text        is_comment

 status1                 0
 status2                 0
 statusN                 0
comment1           status1
comment2           status1
commentN           statusN

Answers


Why do you need this to be in a CSV? It is already structured and ready to be persisted as JSON.

If you really need the tabular approach offered by CSV, then you have to either denormalize it, or use more than one CSV table with references from one to another (and again, the best approach would be to put the data in an SQL database which takes care of the relationships for you)

That said, the way to denormalize is simply to save the same status text to each row where a comment is - that is: record you CSV row in the innermost loop with your approach:

import csv
status_text = []
comment_text = []
writer = csv.writer(open("mycsv.csv", "wt"))
for s in test_statuses:

    test_messages.append(s['message'])
    for c in s['comments']['data']:
        test_comments.append(c['message'])
        writer.writerow((s['message'], c['message']))

Note that you'd probably be better by writing the status idto each row, and create a second table with the status message where the id is the key (and put it in a database instead of various CSV files). And then, again, you are probably better of simply keeping the JSON. If you need search capabilities, use a JSON capable database such as MongoDB or PostgreSQL


Need Your Help

How to Copy a file to another directory after submit

php function copy

I am trying to create a directory after a form submit, then after the form is submitted I want that directory to have a file stuffed inside of it. I have tried the php copy function and used an if

PHP/JS: Echoing several variables without losing their value

php javascript jquery ajax

The JS/Ajax function I have built submits without a button click or page refresh. The function gets the values of the input field and with php echoed out the results. But everytime a variable is ec...