Federated Learning (FL) and Multi-agent Reinforcement Learning (MARL) are two emerging machine learning paradigms for future intelligent wireless IoT and networked systems. FL is a data-driven supervised machine learning setting where the centralized location trains a learning model by using remote devices (e.g., sensors, user devices). On the other hand, the decentralized MARL schemes, which are based on interactions of the learning agents with the environment, present suitable frameworks to solve decision and control problems considering the heterogeneity of IoT systems. In this talk, I shall discuss example applications and also the challenges of employing FL and MARL methods in resource-constrained and unreliable wireless IoT systems and networks. I shall present an FL algorithm that is suitable for a resource-constrained wireless access network and also a MARL method for a practical wireless edge computing environment. To this end, I shall discuss several of the key open research issues.