Changing style of pandas.DataFrame: Permanently?

Question

When I change the style of a pandas.DataFrame, for instance like so

        # color these columns
        color_columns = ['roi', 'percent_of_ath']
        (portfolio_df
            .style
            # color negative numbers red
            .apply(lambda v: 'color: red' if v < 0 else 'color: black',
                   subset=color_columns)
            # color selected cols light blue
            .apply(lambda s: 'background-color: lightblue',
                    subset=color_columns))

the styles applied to the dataframe are not permanent.

To make them stick I can assign the output of the (portfolio_df ... part to the same dataframe like so:

portfolio_df = (portfolio_df ...

Displaying this overwritten portfolio_df in a Jupyter Notebook, I can see the beautifully styled DataFrame. But trying to change the style from within a function that is imported from a module, I fail. I construct the DataFrame in the function, change the style, return the (now) styled DataFrame from the function, display it in the Jupyter Notebook, I see a non-styled DataFrame.

Edit

Inspecting the type of the return value of the styling operation

s = (portfolio_df.style.apply(...

I see this:

>>> type(s)
pandas.io.formats.style.Styler

So the operation does not return a DataFrame, but a ...Styler object. I was erroneously thinking that I can re-assign this return value to my original DataFrame, thus overwrite it and make the style change permanent.

Question

Is the operation of applying a style to a DataFrame a destructive or non-desctructive operation? The answer seems to be that the style is not changed permanently. Now, how can I make it change permanently?

Edit 2

Viewing the source code of Pandas, I looked at the docstring for class Styler (see [1]):

    If using in the Jupyter notebook, Styler has defined a ``_repr_html_``
    to automatically render itself. Otherwise call Styler.render to get
    the generated HTML.

So in a Jupyter notebook, Styler has a method that auto renders the dataframe, respecting the applied style.

Otherwise (in iPython) it creates HTML.

Assigning the return value of the applied style to a variable

s = (portfolio_df.style.apply(...

I can use it in an Jupyter notebook to render the new style.

What I understand is this: I cannot output my dataframe into a Jupyter notebook and expect it to render the new style. But I can output s to show the new style.

---

[1] class Styler in

pandas/pandas/io/formats/style.py

Docstring, line 39.

Salita

Please edit your post to include the full traceback of the error – Vitriolic

I have a similar use case, in which I want to apply a style and convert to latex. – Carnivorous

Could you elaborate But trying to change the style from within a function that is imported from a module, I fail. with your code? – Instructor

Answer 1

I can give you two recommendations:

1. Write a simple function to display your dataframes

This is by far the simplest and least hacky solution. You could write:

def my_style(df:pd.DataFrame, color_columns:list[str]=['roi', 'percent_of_ath']):
    return (df
            .style
            .applymap(lambda v: 'color: red' if v < 0 
                                 else None, subset=color_columns)
           )    

This lets you write code like:

df.pipe(my_style) # This will output a formatted dataframe

Or

from IPython.display import display 

# This will print a nicely formatted dataframe
def my_display(df:pd.DataFrame, style=my_style):
    display(df.pipe(style))

2. Overwrite the Pandas _repr_html_ method

I don't advice this, but it is what you are asking for ;)

from pandas._config import get_option
from pandas.io.formats import format as fmt

def _my_repr_html_(self) -> str | None:
        """
        Return a html representation for a particular DataFrame.

        Mainly for IPython notebook.
        """
        if self._info_repr():
            buf = StringIO()
            self.info(buf=buf)
            # need to escape the <class>, should be the first line.
            val = buf.getvalue().replace("<", r"&lt;", 1)
            val = val.replace(">", r"&gt;", 1)
            return "<pre>" + val + "</pre>"

        if get_option("display.notebook_repr_html"):
            max_rows = get_option("display.max_rows")
            min_rows = get_option("display.min_rows")
            max_cols = get_option("display.max_columns")
            show_dimensions = get_option("display.show_dimensions")

            formatter = fmt.DataFrameFormatter(
                self,
                columns=None,
                col_space=None,
                na_rep="NaN",
                formatters=None,
                float_format=None,
                sparsify=None,
                justify=None,
                index_names=True,
                header=True,
                index=True,
                bold_rows=True,
                escape=True,
                max_rows=max_rows,
                min_rows=min_rows,
                max_cols=max_cols,
                show_dimensions=show_dimensions,
                decimal=".",
            )
            # return fmt.DataFrameRenderer(formatter).to_html(notebook=True)
            return self.pipe(my_style).to_html(notebook=True) # <<<< !!! HERE !!! 
        else:
            return None
        
df.pipe(_my_repr_html_)

pd.DataFrame._repr_html_ = _my_repr_html_

Be careful! This sample code does not handle very long or wide DataFrames.

Edit:

The code above for overwriting repr_html has a minimal edit of the pandas code. This is a minimal working example:

def my_style(df:pd.DataFrame, color_columns:list[str]=['roi', 'percent_of_ath']):
    return (df.style.applymap(
            lambda v: 'color: red' if v < 0 else None, subset=color_columns)
           ) 

def _my_repr_html_(self) -> str | None:
    return self.pipe(my_style)._repr_html_() # <<<< !!! HERE !!! 
        
pd.DataFrame._repr_html_ = _my_repr_html_

Superconductivity

Answer 2

I put a startup script for my pandas options in my ipython profile.

For example here is what my ipython startup looks like

$ ls ~/.ipython/profile_default/startup/
10-pandas.py  11-import-modules.py  README

My pandas setup looks like

def setup_pandas_options(max_rows=50):
    import pandas as pd
    pd.set_option('display.expand_frame_repr', True)
    pd.set_option('display.max_rows', max_rows)
    pd.set_option('display.max_columns', 100)
    pd.set_option('display.max_colwidth',180)
    pd.set_option('display.width', 200)

Then my inports startup looks like:

try:
    import os
    import sys
    import numpy as np
    import pandas as pd
    setup_pandas_options()
except ModuleNotFoundError:
    pass

You may need to setup a profile initially.

Chinchilla

Answer 3

This answer ([edit] or "hack") applies in case you need specific objects to be printed in a custom way, as opposed to all dataframe objects. It uses together the "customise _repr_html_" approach (not too different from what featured in other answers here) and the "override bound method" one (e.g. https://mcmap.net/q/174182/-override-a-method-at-instance-level).

Test data:

import pandas
portfolio_df = pandas.DataFrame({
    'schnibble':      [  5,  9, 11 ],
    'roi':            [ -4, 19, -3 ],
    'percent_of_ath': [ 10, 30, 50 ],
})
portfolio_df

A self-standing function needs to be defined, which applies the customisation to self.style and then delegates the rest to the _repr_html_ method of the resulting object.

def paintPortfolioDF(df):
    # color these columns
    color_columns = ['roi', 'percent_of_ath']
    return (df
        .style
        # color negative numbers red
        .map(lambda v: 'color: red' if v < 0 else 'color: black',
               subset=color_columns)
        # color selected cols light blue
        .map(lambda s: 'background-color: lightblue',
                subset=color_columns)
    )._repr_html_()
portfolio_df._repr_html_ = paintPortfolioDF.__get__(portfolio_df, None)

(note that I had to replace apply with map to make the original statement work with my setup).

Note: while this may be exactly what you need (at least it was exactly what I needed), it comes with all the side effects of overriding a method in a class instance (as opposed to the class itself). This is especially relevant in tools like Pandas that encourage patterns like:

portfolio_df = portfolio_df.rename({ 'schnibble': 'Schnibble' })

The return value of portfolio_df.rename is a new DataFrame object that does not preserve the customisation of the original object, so the instruction above effectively removes that customisation (it can be restored with another portfolio_df._repr_html_ = paintPortfolioDF.__get__(portfolio_df, None) as long as paintPortfolioDF is still around).

---

Test setup: Python 3.12.4, IPython 8.26.0, Jupyter notebook 7.2.1, pandas 2.2.2.

Lathrope

Answer 4

try using this function

df.style.applymap()

Lenity