process large text file in python

Assuming that the structure of each entry will always be the same, just do something like this:

import csv

# Open the file
f = open("/path/to/large.file", "r")
# Create an output file
output_file = open("/desired/path/to/final/file", "w")

# Use the CSV module to make use of existing functionality.
final_file = csv.writer(output_file)

# Write the header row - can be skipped if headers not needed.
final_file.writerow(["LoginID","EmailAddress"])

# Set up our temporary cache for a user
current_user = []

# Iterate over the large file
# Note that we are avoiding loading the entire file into memory
for line in f:
    if line.startswith("LoginID"):
        current_user.append(line[9:].strip())
    # If more information is desired, simply add it to the conditions here
    # (additional elif's should do)
    # and add it to the current user.

    elif line.startswith("mail"):
        current_user.append(line[6:].strip())
        # Once you know you have reached the end of a user entry
        # write the row to the final file
        # and clear your temporary list.
        final_file.writerow(current_user)
        current_user = []

    # Skip lines that aren't interesting.
    else:
        continue

Again assuming your file is well-formed:

with open(inputfilename) as inputfile, with open(outputfilename) as outputfile:
    mail = loginid = ''
    for line in inputfile:
        line = inputfile.split(':')
        if line[0] not in ('LoginId', 'mail'):
            continue
        if line[0] == 'LoginId':
            loginid = line[1].strip()
        if line[0] == 'mail':
            mail = line[1].strip()
        if mail and loginid:
            output.write(loginid + ',' + mail + '\n')
            mail = loginid = ''

Essentially equivalent to the other methods.

To open the file you'll want to use something like the with keyword to ensure it closes properly even if something goes wrong:

with open(<your_file>, "r") as f:
   # Do stuff

As for actually parsing out that information, I'd recommend building a dictionary of ID email pairs. You'll also need a variable for the uid and the email.

data = {}
uid = 0
email = ""

To actually parse through the file (the stuff run while your file is open) you can do something like this:

for line in f:
    if "uid=" in line:
        # Parse the user id out by grabbing the substring between the first = and ,
        uid = line[line.find("=")+1:line.find(",")]
    elif "mail:" in line:
        # Parse the email out by grabbing everything from the : to the end (removing the newline character)
        email = line[line.find(": ")+2:-1]
        # Given the formatting you've provided, this comes second so we can make an entry into the dict here
        data[uid] = email

Using the CSV writer (remember to import csv at the beginning of the file) we can output like this:

writer = csv.writer(<filename>)
writer.writerow("User, Email")
for id, mail in data.iteritems:
    writer.writerow(id + "," + mail)

Another option is to open the writer before the file, write the header, then read the lines from the file at the same time as writing to the CSV. This avoids dumping the information into memory, which might be highly desirable. So putting it all together we get

writer = csv.writer(<filename>)
writer.writerow("User, Email")
with open(<your_file>, "r") as f:
    for line in f:
        if "uid=" in line:
            # Parse the user id out by grabbing the substring between the first = and ,
            uid = line[line.find("=")+1:line.find(",")]
        elif "mail:" in line:
            # Parse the email out by grabbing everything from the : to the end (removing the newline character)
            email = line[line.find(": ")+2:-1]
            # Given the formatting you've provided, this comes second so we can make an entry into the dict here
            writer.writerow(iid + "," + email)