dataloader Questions
2
Solved
I am using densenet121 to do cat/dog detection from Kaggle dataset. I enabled cuda and it appears that training is very fast. However, the data loading (or perhaps processing) appears to be very sl...
Pulido asked 23/4, 2020 at 17:40
4
Solved
In training loop, I load a batch of data into CPU and then transfer it to GPU:
import torch.utils as utils
train_loader = utils.data.DataLoader(train_dataset, batch_size=128, shuffle=True, num_wo...
Arris asked 31/5, 2020 at 4:30
2
Solved
I have the following code:
import torch
import numpy as np
import pandas as pd
from torch.utils.data import TensorDataset, DataLoader
# Load dataset
df = pd.read_csv(r'../iris.csv')
# Extract fea...
Probationer asked 24/6, 2020 at 7:34
2
Solved
I am trying to train a pretrained roberta model using 3 inputs, 3 input_masks and a label as tensors of my training dataset.
I do this using the following code:
from torch.utils.data import TensorD...
Rizzio asked 13/12, 2020 at 18:23
2
Solved
I'm trying to initialize an instance of DataLoader using the following code:
const authorLoader = new DataLoader(async (keys:string[]) => {
// Return an author for each book
});
I'm getting ...
Peaslee asked 27/5, 2020 at 20:10
4
Solved
I have a torch.utils.data.DataLoader. I have created them with the following code.
transform_train = transforms.Compose([
transforms.RandomCrop(32, padding=4),
transforms.RandomHorizontalFlip(),
...
Mittel asked 28/3, 2022 at 5:10
2
Solved
using React-Router V6 and I tried to strongly type the dataloader functions that are using params and also the useLoaderData hook.
So far I have to do the following that is ugly :
A- For useLoaderD...
Sandhi asked 21/12, 2022 at 13:58
5
Solved
In my Flutter project, I am doing API calls to fetch data by GET request. After parsing the JSON object from the response, I just show the value in the Text widget. While the data takes time to loa...
Tocology asked 23/7, 2019 at 14:44
1
Solved
I have a "torch.utils.data.DataLoader". I want to rearrange the order of the samples. Is it possible?
Joacimah asked 27/3, 2022 at 22:0
2
Solved
I'm trying to make a simple image classifier using PyTorch.
This is how I load the data into a dataset and dataLoader:
batch_size = 64
validation_split = 0.2
data_dir = PROJECT_PATH+"/categorized_...
Cozmo asked 11/6, 2020 at 7:32
2
Solved
Suppose I have a list, datalist which contains several examples (which are of type torch_geometric.data.Data for my use case). Each example has an attribute num_nodes
For demo purpose, such datalis...
Hamfurd asked 26/1, 2022 at 14:17
2
Solved
There is a bug in PyTorch/Numpy where when loading batches in parallel with a DataLoader (i.e. setting num_workers > 1), the same NumPy random seed is used for each worker, resulting in any rand...
Cause asked 20/4, 2021 at 14:29
3
I have two dataloaders and I would like to merge them without redefining the datasets, in my case train_dataset and val_dataset.
train_loader = DataLoader(train_dataset, batch_size = 512, drop_last...
Dacha asked 7/1, 2021 at 23:29
2
Solved
I'm working on an NLP problem and am using PyTorch. For some reason, my dataloader is returning malformed batches. I have input data that comprises sentences and integer labels.
The sentences can e...
Avuncular asked 17/11, 2020 at 22:2
2
Solved
Is there a way to load a pytorch DataLoader (torch.utils.data.Dataloader) entirely into my GPU?
Now, I load every batch separately into my GPU.
CTX = torch.device('cuda')
train_loader = torch.util...
Deleterious asked 16/12, 2020 at 16:26
1
Solved
There are several scenarios that make me confused about shuffling the data loader, which are as follows.
I set the “shuffle” parameter to False on both train_loader and valid_loader. then the resul...
Accuracy asked 22/12, 2020 at 3:16
0
I'm training my neural network using PyTorch framework. The data is full HD images (1920x1080). But in each iteration, I just need to crop out a random 256x256 patch from these images. My network i...
Pretor asked 29/11, 2020 at 7:27
1
I'm trying to figure out the best way to handle a one-to-many relationship using type-graphql and typeorm with a postgresql db (using apollo server with express). I have a user table which has a on...
Rutter asked 30/8, 2020 at 23:42
1
I have the following pieces of code:
# Device configuration
device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
seed = 42
np.random.seed(seed)
torch.manual_seed(seed)
# split t...
Monocotyledon asked 14/10, 2020 at 17:8
1
Solved
hopefully someone can help me with this little problem, I just cannot figure it out right now.
Problem Statement:
I want to access 'context' for the sake of authentication in my DataLoader. This Da...
Hibernia asked 1/10, 2020 at 14:46
2
Solved
I have a question that How to get the total number of batch iteration from pytorch dataloader?
The following is a common code for training
for i, batch in enumerate(dataloader):
Then, is there any...
Claque asked 17/9, 2020 at 3:0
2
Solved
When I create a PyTorch DataLoader and start iterating -- I get an extremely slow first epoch (x10--x30 slower then all next epochs). Moreover, this problem occurs only with the train dataset from ...
Goof asked 30/8, 2020 at 5:26
0
PyTorch Dataloader hangs when num_workers > 0. The code hangs with only about 500 M GPU memory usage.
System info: NVIDIA-SMI 418.56 Driver Version: 418.56 CUDA Version: 10.1.
The same issue app...
Interfluve asked 31/8, 2020 at 15:54
3
How to load entire dataset from the DataLoader? I am getting only one batch of dataset.
This is my code
dataloader = torch.utils.data.DataLoader(dataset=dataset, batch_size=64)
images, labels = n...
Withstand asked 7/8, 2019 at 4:13
2
I'm using a couple of dataloaders that use injected query services (which in turn have dependencies on a DbContext). It looks something like this:
Field<ListGraphType<UserType>>(
"Use...
Corliss asked 15/11, 2019 at 8:45
1 Next >
© 2022 - 2025 — McMap. All rights reserved.