Create new columns and fill with calculated values from same dataframe

Question

Here is a simplified example of my df:

ds = pd.DataFrame(np.abs(randn(3, 4)), index=[1,2,3], columns=['A','B','C','D'])
ds['sum'] = ds.sum(axis=1)

which looks like

      A         B         C         D       sum
1  0.095389  0.556978  1.646888  1.959295  4.258550
2  1.076190  2.668270  0.825116  1.477040  6.046616
3  0.245034  1.066285  0.967124  0.791606  3.070049

I would like to create 4 new columns and calculate the percentage value from the total (sum) in every row. So first value in the first new column should be (0.095389/4.258550), first value in the second new column (0.556978/4.258550)...and so on.

Eloiseloisa

Answer 1

You can do this easily manually for each column like this:

df['A_perc'] = df['A']/df['sum']

---

If you want to do this in one step for all columns, you can use the div method (http://pandas.pydata.org/pandas-docs/stable/basics.html#matching-broadcasting-behavior):

ds.div(ds['sum'], axis=0)

And if you want this in one step added to the same dataframe:

>>> ds.join(ds.div(ds['sum'], axis=0), rsuffix='_perc')
          A         B         C         D       sum    A_perc    B_perc  \
1  0.151722  0.935917  1.033526  0.941962  3.063127  0.049532  0.305543   
2  0.033761  1.087302  1.110695  1.401260  3.633017  0.009293  0.299283   
3  0.761368  0.484268  0.026837  1.276130  2.548603  0.298739  0.190013   

     C_perc    D_perc  sum_perc  
1  0.337409  0.307517         1  
2  0.305722  0.385701         1  
3  0.010530  0.500718         1  

Mercia

When doing something like this, does pandas guarantee the row ordering will not change? – Donohoe

Yes, the row ordering is certainly preserved. – Mercia

Answer 2

In [56]: df = pd.DataFrame(np.abs(randn(3, 4)), index=[1,2,3], columns=['A','B','C','D'])

In [57]: df.divide(df.sum(axis=1), axis=0)
Out[57]: 
          A         B         C         D
1  0.319124  0.296653  0.138206  0.246017
2  0.376994  0.326481  0.230464  0.066062
3  0.036134  0.192954  0.430341  0.340571

Excrete

For clarity, div and divide are the same (just aliases) – Mercia

Answer 3

You can convert sum column in a numpy column array and broadcast division.

new_df = df / df[['sum']].values     # note the double-brackets around 'sum'

To add the percentages as new columns,

df[df.columns.drop('sum') + '_perc'] = df.drop(columns='sum') / df[['sum']].values

Expectation