I love the idea of framework, but remains a “work machine” for me, not for fidgetting, distrohopping, hobby, etc.
I want a “clean solution”, not “a pile workarounds” because usually it severely impact productivity.
However, thank for the link.
Why not just carry CPU and memory in your pocket instead. Who needs these fancy peripherals. Disk? USB stick. Keyboard? Neuralink.
I use my machines for work too. The 16" Laptop is a bit too big for me to carry around though. So any place I take it would be a stationary situation.
I wouldn’t really consider connecting a portable AI accelerator which can fit into your bag as “fidgeting” or “distro hopping” or any of that other hobbiest stuff. This is plugging in an accelerator for your specialized workload.
That makes no sense at all either. A laptop is a laptop which is a fully functioning machine. If you are doing specific workloads that need an accelerator and there is a small lightweight accelerator which you can put into your bag… I’m not sure how this is anything like carrying around an entire computer of parts that is not assembled in your pocket.
I’m not sure how this is anything like carrying around an entire computer of parts that is not assembled in your pocket
That’s the point just assemble them together. I was being generous with memory anyway
yes but if you have a specialized workload (especially if this isn’t the only thing you do) then having an attachment that can be connected is perfectly reasonable.
But do we know any better accelerator than the 4GB nvidia. I cannot imagine having less than 16 and I do my AI stuff in the cloud usually.
Not in a small portable package like that. NVIDIA is very specific about where their GPUs can go and how they can be used. So it’s already kind of lucky that this type of device was allowed to be made.
Fuck nvidia
2 Likes
I agree in this regard. They are a very controlling anti-consumer company. However lots of AI stuff today is setup to just run on Nvidia hardware. Now with ROCm there are ways to run it all on AMD but the translation layer has more work to be done on the consumer side.
I get it. I am even able to admit that Nvidia has done a lot for AI development. That still does not excuse them.
It doesn’t and it is why I am so excited for ROCm.
EDIT2:
Sigh. Just see next post. Sorry
EDIT:
I spoke to soon, looks like it doesn’t work. ROCblas doesn’t seem to support this GPU. Well that sucks.
Sorry for the crazy delay. I tried with Jax but the rocm stuff was a pain in the butt. Switched back to pytorch though and edited: well, the GPU is visible but doesn’t work when you try to use tensor ops.
>>> torch.cuda.is_available()
True
>>> torch.cuda.device_count()
2
>>> torch.cuda.device(0)
<torch.cuda.device object at 0x73ffb835b920>
>>> torch.cuda.get_device_properties(0)
_CudaDeviceProperties(name='AMD Radeon™ RX 7700S', major=11, minor=0, gcnArchName='gfx1102', total_memory=8176MB, multi_processor_count=16)
>>> torch.cuda.get_device_properties(1)
_CudaDeviceProperties(name='AMD Radeon Graphics', major=11, minor=0, gcnArchName='gfx1103', total_memory=512MB, multi_processor_count=6)
What an emotional rollercoaster 
I was able to get it to work (at least for the MNIST torchvision example).
Train Epoch: 14 [58240/60000 (97%)] Loss: 0.021353
Train Epoch: 14 [58880/60000 (98%)] Loss: 0.003511
Train Epoch: 14 [59520/60000 (99%)] Loss: 0.001184
Test set: Average loss: 0.0262, Accuracy: 9914/10000 (99%)
The trick was HSA_OVERRIDE_GFX_VERSION=11.0.0 python main.py
2 Likes
Thank you for testing and reporting in!
Well, on one side it is definitely encouraging seeing that potentially it can even run on the integrated AMD graphics, mimicking perhaps the power of an embedded device.
But yes, the ROCm support doesn’t seem as mature as the CUDA counterpart, probably it’s a bit of a catch 22, given it has much less users to test it.
Plus it’s own charger, cables, probably some issues with the drivers … I mean, don’t get me wrong, I’m happy it exists, but perhaps having it built-in makes things so much easier.
Yes, I’ll see if I can come up with a reasonable benchmark this weekend but that was definitely more frustrating than I’d hoped for – but I’m pretty excited now that I have it working.
No, the whole thing runs off of a single cable with Thunderbolt provided that the laptop can provide enough power to the GPU. If not, it has a secondary USB-C input.
It is a newer software that only came out in the last year, so it’s definitely not going to be as good as a software that has been around for 20 years, but it is rapidly improving.
Absolutely, and I’m definitely looking forward to it. However already today, with good old tested CUDA just running research paper code it’s often a bit of a nightmare, let’s say 50% of the time. I do not wish to add an even weaker layer for GPU compatibility.