The Industrial Internet of Things (IIoT) has played a key role in enabling an efficient and interconnected industry through real-time communication and processing systems, thereby building on the principles of Industry 4.0. Nowadays, industrial systems are in the process of transitioning towards Industry 5.0, where humans will once again take center stage in decision-making, supported by Artificial Intelligence (AI)-based methods. In this context, drones have emerged as a feasible device for enhancing environmental sensing tasks, reducing operational costs, alleviating communication bottlenecks, and cooperating with humans through the use of Virtual Reality (VR) and Augmented Reality (AR) platforms. Therefore, Internet of Drones (IoD) network paradigm has been adopted in the industry, giving rise to the Industrial Internet of Drones (IIoD). Given these aspects, there have been changes in the privacy and security requirements for this network environment, which demands a thorough analysis of these modifications, including the challenges that arise and possible solutions to overcome them. Consequently, this study analyzes the privacy and security issues related to IIoD. Namely, we highlight the elements of IIoD which require protection, the threats and the countermeasures. We also present how these aspects differ from the general IoD environment. Lastly, we discuss the challenges regarding IIoD security and privacy, leveraging the future directions to address Industry 5.0 aspects.