missing-data Questions
2
Solved
The gnuplot command set datafile missing "nan" tells gnuplot to ignore nan data values in the data file.
How to ignore both nan and -nan? I tried the following in gnuplot, but then the effect of t...
Fungoid asked 10/9, 2013 at 11:44
4
Solved
I am trying to replace some missing values in my data with the average values from a similar group.
My data looks like this:
X Y
1 x y
2 x y
3 NA y
4 x y
And I want it to look like this:
X ...
Pulpboard asked 13/7, 2011 at 19:47
10
Solved
The following code does not work.
import pandas as pd
import numpy as np
df=pd.DataFrame(['ONE','Two', np.nan],columns=['x'])
xLower = df["x"].map(lambda x: x.lower())
How should I tweak it to...
Givens asked 7/3, 2014 at 8:34
14
Solved
I am not sure how to loop over each column to replace the NA values with the column mean. When I am trying to replace for one column using the following, it works well.
Column1[is.na(Column1)] <...
Thurible asked 14/9, 2014 at 16:50
4
Solved
I have huge matrix with a lot of missing values. I want to get the correlation between variables.
1. Is the solution
cor(na.omit(matrix))
better than below?
cor(matrix, use = "pairwise.comple...
Tremble asked 16/9, 2011 at 13:44
4
I have the following data structure. Within each group, some observations have missing value. I do know that each group has only one non-missing value (10 for group 1 and 11 for group 2 in this cas...
Alexina asked 2/12, 2015 at 20:53
5
Solved
I've been goggling for hours and I'm not sure where to find the answer for something simple like this, so I hope this is not a duplicate question.
I have a large data frame (936848 x 12) with one c...
Heptode asked 21/3, 2022 at 8:19
2
I want to do a rolling computation on missing data.
Sample Code: (For sake of simplicity I'm giving an example of a rolling sum but I want to do something more generic.)
foo = lambda z: z[pandas....
Rephrase asked 15/11, 2012 at 20:27
5
Solved
I am trying to learn pandas but I have been puzzled with the following. I want to replace NaNs in a DataFrame with the row average. Hence something like df.fillna(df.mean(axis=1)) should work but f...
Adore asked 10/10, 2015 at 20:21
6
Solved
I have a dataset like this where data for some years are missing .
County Year Pop
12 1999 1.1
12 2001 1.2
13 1999 1.0
13 2000 1.1
I want something like
County Year Pop
12 1999 1.1
12 2000 NaN...
Interlunation asked 6/5, 2017 at 4:26
4
There appears to be a quirk with the pandas merge function. It considers NaN values to be equal, and will merge NaNs with other NaNs:
>>> foo = DataFrame([
['a',1,2],
['b',4,5],
['c',7...
Loyce asked 29/5, 2014 at 18:17
4
Solved
I am trying to run a lme model with these data:
tot_nochc=runif(10,1,15)
cor_partner=factor(c(1,1,0,1,0,0,0,0,1,0))
age=runif(10,18,75)
agecu=age^3
day=factor(c(1,2,2,3,3,NA,NA,4,4,4))
dt=as.data....
Reaganreagen asked 7/7, 2016 at 16:12
4
Solved
I have given the following df
df = pd.DataFrame(data = {'day': [1, 1, 1, 2, 2, 3], 'pos': 2*[1, 14, 18], 'value': 2*[1, 2, 3]}
df
day pos value
0 1 1 1
1 1 14 2
2 1 18 3
3 2 1 1
4 2 14 2
5 3 18 ...
Hydrophyte asked 7/10, 2020 at 14:43
6
Solved
I have a data frame with many columns and many rows.
col_1 | col_2 | ... | col_n
35 | NA | ... | 2
. | . | . | .
. | . | . | .
. | . | . | .
123 | 90 | ... | NA
Some rows contain NA values (c...
Shortcake asked 5/6, 2021 at 13:54
1
Solved
Pandas Docs for pandas.Series.map says that:
"When arg is a dictionary, values in Series that are not in the dictionary (as keys) are converted to NaN. However, if the dictionary is a dict sub...
Boulware asked 3/6, 2021 at 18:4
3
Solved
How can I randomly insert np.nan's in a DataFrame ?
Let's say I want 10% null values inside my DataFrame.
My data looks like this :
df = pd.DataFrame(np.random.randn(5, 3),
index=['a', 'b', '...
Ungrounded asked 20/8, 2016 at 14:48
3
Solved
Suppose I have a pandas.DataFrame called df. The columns of df represent different individuals and the index axis represents time, so the (i,j) entry is individual j's observation for time period i...
Offside asked 14/8, 2013 at 14:0
8
Solved
I am trying to print or to get list of columns name with missing values. E.g.
data1 data2 data3
1 3 3
2 NaN 5
3 4 NaN
I want to get ['data2', 'data3'].
I wrote following code:
print('\n'.j...
Pervert asked 21/5, 2016 at 18:37
1
In related to question posted in One Hot Encoding preserve the NAs for imputation I am trying to create a custom function that handles NAs when one hot encoding categorical variables. The setup sho...
Topography asked 15/3, 2021 at 9:2
9
I'm working on a machine learning problem in which there are many missing values in the features. There are 100's of features and I would like to remove those features that have too many missing va...
Pampero asked 4/8, 2017 at 20:30
5
I am new to R and struggling with a problem.
I need a function to impute the missing values in a vector according to the mean value of the elements within a window of a given size.
However, this ...
Triplicity asked 6/2, 2017 at 16:30
4
Solved
I am trying to exclude rows have missing values (NA) in all columns for that row AND for which all subsequent rows have only missing values (or is the last empty row itself), i.e. I want to remove ...
Falsehood asked 12/1, 2021 at 17:20
6
Solved
While working in Pandas in Python...
I'm working with a dataset that contains some missing values, and I'd like to return a dataframe which contains only those rows which have missing data. Is the...
Alyss asked 25/5, 2015 at 23:3
2
Solved
I have the following pandas dataframe and I would like to fill the NaNs in columns A-C in a row-wise fashion with values from columns D. Is there an explicit way to do this where I can define that ...
Eurus asked 3/6, 2014 at 12:37
2
Solved
I have couple of date columns, I want to convert them to month/day/year format. Let's say test is one of the date columns - below code works.
dfq['test1'] = dfq['test1'].apply(lambda x: x.strftime...
Ajit asked 23/4, 2018 at 22:4
© 2022 - 2025 — McMap. All rights reserved.