I read data from a .csv file to a Pandas dataframe as below. For one of the columns, namely id, I want to specify the column type as int. The problem is the id series has missing/empty values. Whe...

I would like to import the following csv as strings not as int64. Pandas read_csv automatically converts it to int64, but I need this column as string. ID 00013007854817840016671868 000130078548178...

I've been having trouble dealing with no data values in Python's rasterio package when applying a polygon mask on a raster data set. This particular raster is Landsat uint8 with 7 bands and the no ...

I'm working with the arrow dtypes with pandas, and my dataframe has a variable that should be categorical, but I can't figure out how to transform it into pyarrow data type for categorical data (di...

I am trying to plot a trendline for my data. However, I am getting the error ValueError: data type <class 'numpy.object_'> not inexact. Can someone explain why? my dataframe is Us_corr3; H...

I have a column in my dataframe that contains nan values and int values. The original dType was float64, but I was trying to change it to int6, and change nan values to np.nan. now I get this error...

If I have a dataframe with the following columns: 1. NAME object 2. On_Time object 3. On_Budget object 4. %actual_hr float64 5. Baseline Start Date datetime64[ns] 6. Forecast Start Date datetime64[...

Is there a generic way to convert all float64 values in a pandas dataframe to float32 values? But not changing uint16 to float32? I don't know the signal names in advance but just want to have no f...

I have a data frame with alpha-numeric keys which I want to save as a csv and read back later. For various reasons I need to explicitly read this key column as a string format, I have keys which ar...

How to optimize the data frame memory footprint and find the most optimal (minimal) data types dtypes for numeric columns. For example: A B C D 0 1 1000000 1.1 1.111111 1 2 -1000000 2.1 2.111111 ...

Float32 vs float32? What is the purpose of uppercase vs lowercase dtypes in Pandas? Uppercase seems more error prone: TypeError: object cannot be converted to a FloatingDtype. dtype = { 'doom_floa...

Let df_1 and df_2 be: In [1]: import pandas as pd ...: df_1 = pd.DataFrame({'a': [1, 2, 3], 'b': [4, 5, 6]}) ...: df_2 = pd.DataFrame({'a': [1, 2, 3], 'b': [4, 5, 6]}) In [2]: df_1 Out[2]: a b ...

I am new to pandas. I have a column with the object dtype that contains floats and strings. I would like to count the number of each in the column. I figured out a way to do it: len([v for v in lis...

I have a pd.DataFrame which contains different dtypes columns. I would like to have the count of columns of each type. I use Pandas 0.24.2. I tried: dataframe.dtypes.value_counts() It worked f...

I am reading in a huge fixed width text file in chunks and export the data as csv. Because pandas.read_fwf does not allow to specify the dtypes, I am wondering what other way there exists to force ...