Implementation of Python Pandas list List Data Column Splitting into Multiple Rows


1. The effect achieved

Sample code:

df=pd.DataFrame({'A':[1,2],'B':[[1,2],[1,2]]})
df
Out[458]:
  A    B
0 1 [1, 2]
1 2 [1, 2]

Effect of splitting into multiple lines:

A B 0 1 1 1 1 2 3 2 1 4 2 2

2. The method of splitting into multiple lines

  1. Implemented through apply and pd. Series

Easy to understand, but not recommended in terms of performance.

df.set_index('A').B.apply(pd.Series).stack().reset_index(level=0).rename(columns={0:'B'})
Out[463]:
  A B
0 1 1
1 1 2
0 2 1
1 2 2
  1. Using the repeat and DataFrame constructors

Performance is OK, but it is not suitable for multiple columns

df=pd.DataFrame({'A':df.A.repeat(df.B.str.len()),'B':np.concatenate(df.B.values)})
df
Out[465]:
  A B
0 1 1
0 1 2
1 2 1
1 2 2

Or

s=pd.DataFrame({'B':np.concatenate(df.B.values)},index=df.index.repeat(df.B.str.len()))
s.join(df.drop('B',1),how='left')
Out[477]:
  B A
0 1 1
0 2 1
1 1 2
1 2 2
  1. Create a new list
pd.DataFrame([[x] + [z] for x, y in df.values for z in y],columns=df.columns)
Out[488]:
  A B
0 1 1
1 1 2
2 2 1
3 2 2

Or

# Split into more than two columns
s=pd.DataFrame([[x] + [z] for x, y in zip(df.index,df.B) for z in y])
s.merge(df,left_on=0,right_index=True)
Out[491]:
  0 1 A    B
0 0 1 1 [1, 2]
1 0 2 1 [1, 2]
2 1 1 2 [1, 2]
3 1 2 2 [1, 2]
  1. Implementation using reindex and loc
df.reindex(df.index.repeat(df.B.str.len())).assign(B=np.concatenate(df.B.values))
Out[554]:
  A B
0 1 1
0 1 2
1 2 1
1 2 2
#df.loc[df.index.repeat(df.B.str.len())].assign(B=np.concatenate(df.B.values)
  1. High performance implementation using numpy
newvalues=np.dstack((np.repeat(df.A.values,list(map(len,df.B.values))),np.concatenate(df.B.values)))
pd.DataFrame(data=newvalues[0],columns=df.columns)
  A B
0 1 1
1 1 2
2 2 1
3 2 2