The Internet of Things (IoT) has grown at a rapid pace in recent years. It requires a large amount of data and massive computational resources, thus the concept of Fog Computing (FC) has emerged. FC attempts to overcome network latency by bringing computational resources closer to IoT devices. One important part of FC is an offloading mechanism to make proper decisions for better utilizing of FC node(s), especially for real-time (low latency and high throughput) applications. Generally, offloading policies are categorized as centralized and distributed. However, by growing numbers of IoT devices which leads to expansion of FC layer beyond the initial configurations, centralized scheduling solutions for time-sensitive tasks suffers from two major challenges: first, increasing complexity, and second, non-fault tolerating. In order to address these issues, scalable decentralized/distributed approaches have been developed to schedule tasks through an autonomous collaboration between a small number of nodes (neighbors). Without a thorough picture of the network or nodes’ state, it is difficult to design algorithms that make optimum decisions. This paper presents a scalable algorithm for offloading time-sensitive tasks through a semi-network aware distributed scheduling mechanism. Based on the evaluation results obtained for acceptance rate, response time, and network resource usage, the proposed method outperforms the state-of-the-art on average.