As Internet of Things (IoT) devices continue to proliferate and evolve, the demand for real-time responses and low latency in IoT applications has intensified. These applications, ranging from smart healthcare to autonomous vehicles, often generate computational tasks that require substantial processing power. Traditional IoT devices with limited resources struggle to meet these demands, leading to high response times and increased energy consumption and cost. Volunteer Edge-Cloud Computing (VECC) offers a promising solution by distributing tasks across edge nodes and cloud servers, thus optimizing resource usage and reducing latency. However, the task offloading problem deciding where and how to process these tasks remains a critical challenge. The challenge is compounded by the dynamic nature of the system, where factors such as node costs, add, leave the system, traffic, and system load continuously fluctuate. Additionally, the system handles a diverse range of tasks, including delay-sensitive tasks that require low latency and computation-intensive tasks demanding substantial processing power. These complexities necessitate an efficient task offloading algorithm capable of determining the optimal execution location for each task type. First, we will present a system model and problem formulation for task offloading within VECC environments using the Multi-Armed Bandit (MAB) theory. In this model, the primary objective is to minimize the total cost, which is composed of two parts: violation cost and monetary cost. We adapt and improve the ε-greedy approach to fit our system model to address the challenges and objectives we previously outlined. This adaptation ensures that the algorithm accounts for the quality of service (QoS) requirements of tasks, focusing on minimizing costs. To address this problem efficiently, we propose (An Adaptive Learning-based Algorithm for IoT Task offloading in Volunteer Edge-Cloud Computing). We will implement the task-specific reward schedules to manage task offloading within the framework of our proposed algorithm, which include critical task scheduling, real-time and normal. Each table will store information specific to its respective task type, which leads to a significant improvement in system performance to effectively handle the tasks. Then we conduct extensive simulation tests to validate and evaluate the performance of our proposed approach. We consider four distinct scenarios and compare our algorithm’s performance with existing algorithms, focusing on the key metrics outlined earlier. Extensive experiments demonstrate that our proposed method significantly outperforms existing approaches and offering a comprehensive and innovative solution to the task offloading challenge in VECC environments.