Update pytorch_data_loader_with_multiple_workers.py#86
Update pytorch_data_loader_with_multiple_workers.py#86ankit-amazon wants to merge 2 commits intoaws-samples:mainfrom
Conversation
Addressed review comments
san2488
left a comment
There was a problem hiding this comment.
I'm not sure this the right fix. I think the issue is that for non-zero num_workers, the dataset should not be stored in a list. So the fix is to not use the list.
Using num_workers as zero is a trivial fix but is likely not what the customer wants.
|
Good point @san2488 I agree with it. It is a compliant case when num_workers = 0. I'd suggest to instead of using datasets.ImageFolder, using a user-defined class and it should have a field which uses numpy array to store image path like DataIter |
|
|
||
| class DataIter(Dataset): | ||
| def __init__(self): | ||
| self.data_np = np.array([x for x in range(24000000)]) |
There was a problem hiding this comment.
can that data be more realistic rather than just numbers? e.g. path to image files in a folder.
improved compliant example