As the number of low cost computing devices at the edge of network increases, there are greater opportunities to enable novel, innovative capabilities, especially in decentralized cyber-physical systems. For example, a set of networked, collaborating processors at the edge can be used to dynamically detect traffic densities via image processing and then use those densities to control the traffic flow by coordinating traffic light sequences; in a decentralized architecture. In this paper we describe a testbed and an application framework for such applications. Furthermore, we describe a queuing theory-based model for analyzing and optimizing workload placement across the fog nodes and available cloud resources.