According to the Ministry of Manpower, falling and slipping accidents are one of the most common accidents in addition, falls from heights (FFH), including accidents during scaffolding work, are still a major cause of death in the construction industry. Regular safety checks are currently being carried out on construction sites, but scaffold-related accidents continue to occur. Sensing technology is being attempted in many industrial sites for safety monitoring, but there are still limitations in terms of the cost of sensors and object detection, which are limited to certain risks. Therefore, this paper proposes a deep learning-based pose estimation approach to identify the risk of falling during scaffolding work in the construction industry. Through analysis of the correlation between unstable behavior during scaffold work and the angle of keypoints of workers, the proposed approach demonstrates the ability to detect the risk of falling. The proposed approach can prevent falling accidents not only by detecting construction site workers, but also by detecting specific risky behaviors. In addition, in limited work environments other than scaffolding work, the information on unstable behavior can be provided to safety managers who may not be aware of the risk, thus contributing to preventing falling accidents