Which of the following descriptions of distributed training is wrong?

Which of the following descriptions of distributed training is wrong?
A . In data parallelism, each computing node trains the complete model based on local data
B . Distributed training can support the training of massive data and complex models
C . Data parallelism and model parallelism are common distributed training methods
D . In model parallelism, each compute node will only interact with the trained sub-model and obtain the final model through model aggregation

View Answer

Answer: D