With technology scaling, the size of cache systems in chip-multiprocessors (CMPs) has been dramatically increased to efficiently store and manipulate a large amount of data in future applications and decrease the gap between cores and off-chip memory accesses. For future CMPs architecting, 3D stacking of LLCs has been recently introduced as a new methodology to combat to performance challenges of 2D integration and the memory wall. However, the 3D design of SRAM LLCs has made the thermal problem even more severe. It, therefore, incurs more leakage energy consumption than conventional SRAM cache architectures in 2Ds due to dense integration. In this paper, we propose two different architectures that exploit the data compression to reduce the energy of LLC and interconnects in 3D-ICs.
In multithreaded applications with high degree of data sharing, the miss rate of private cache is shown to exhibit a compulsory miss component. It manifests because at least some of the shared data originates from other cores and can only be accessed in a shared cache. The compulsory component does not change with the private cache size, causing its miss rate to diminish slower as the cache size grows. As a result, the peak performance of a Chip Multiprocessor (CMP) for workloads with high degree of data sharing is achieved with a smaller private cache, compared to workloads with no data shari…
In this paper is proposed a technique to integrate and simulate a dynamic memory in a multiprocessor framework based on C/C++/SystemC. Using host machine's memory management capabilities, dynamic data processing is supported without compromising speed and accuracy of the simulation. A first prototype in a shared memory context is presented.
Now days, manufacturers are focusing on increasing the concurrency in multiprocessor system-on-a-chip (MPSoC) architecture instead of increasing clock speed, for embedded systems. Traditionally lock-based synchronization is provided to support concurrency; as managing locks can be very difficult and error prone. Transactional memories and lock based systems have been extensively used to provide synchronization between multiple processors [1] in general-purpose systems. It has been shown that locks have numerous shortcomings over transactional memory in terms of power consumption, ease of progr…
Main memories play an important role in overall energy consumption of embedded systems. Using conventional memory technologies in future designs in nanoscale era causes a drastic increase in leakage power consumption and temperature-related problems. Emerging non-volatile memory (NVM) technologies offer many desirable characteristics such as near-zero leakage power, high density and non-volatility. They can significantly mitigate the issue of memory leakage power in future embedded chip-multiprocessor (eCMP) systems. However, they suffer from challenges such as limited write endurance and high…
The next-generation Industrial Internet of Things (IIoT) inherently requires smart devices featuring rich connectivity, local intelligence, and autonomous behavior. Emerging Multiprocessor System-on-Chip (MPSoC) platforms along with comprehensive support for virtualization will represent two key building blocks for smart devices in future IIoT edge infrastructures. We review representative existing solutions, highlighting the aspects that are most relevant for integration in IIoT solutions. From the analysis, we derive a reference architecture for a general virtualization-ready edge IIoT node.…
Power consumption, off-chip memory bandwidth, chip area and Network on Chip (NoC) capacity are among main chip resources limiting the scalability of Chip Multiprocessors (CMP). A closed form analytical solution for optimizing the CMP cache hierarchy and optimally allocating area among hierarchy levels under such constrained resources is developed. The optimization framework is extended by incorporating the impact of data sharing on cache miss rate. An analytical model for cache access time as a function of cache size is proposed and verified using CACTI simulation.
3D integration has the potential to improve the scalability and performance of Chip Multiprocessors (CMP). A closed form analytical solution for optimizing 3D CMP cache hierarchy is developed. It allows optimal partitioning of the cache hierarchy levels into 3D silicon layers and optimal allocation of area among cache hierarchy levels under constrained area and power budgets. The optimization framework is extended by incorporating the impact of multithreaded data sharing on the private cache miss rate. An analytical model for cache access time as a function of cache size and a number of 3D par…
In present study, in order to improve the performance and reduce the amount of power which is dissipated in heterogeneous multicore processors, the ability of detecting the program execution phases is investigated. The programs execution intervals have been classified in different phases based on their throughput and the utilization of the cores. The results of implementing the phase detection technique are investigated on a single core processor and also on a multicore processor. To minimize the profiling overhead, an algorithm for the dynamic adjustment of the profiling intervals is presente…
The trend in industry is towards heterogeneous multicore processors (HMCs), including chips with CPUs and massively-threaded throughput-oriented processors (MTTOPs) such as GPUs. Although current homogeneous chips tightly couple the cores with cache-coherent shared virtual memory (CCSVM), this is not the communication paradigm used by any current HMC. In this paper, we present a CCSVM design for a CPU/MTTOP chip, as well as an extension of the pthreads programming model, called xthreads, for programming this HMC. Our goal is to evaluate the potential performance benefits of tightly coupling he…
The current manufacturing technology allows the integration of a complex multiprocessor system on one piece of silicon (MPSoC for Multiprocessor System-on- Chip). One way to manage the growing complexity of these systems is to increase the level of abstraction and to address the system-level design. In this paper, we focus on the implementation in SystemC language with TLM (Transaction Level Model) to model an MPSOC platform. Our main contribution is to define a comprehensive, fast and accurate method for designing and evaluating performance for MPSoC systems. The studied MPSoC is composed of …
This dissertation develops hardware that automatically reduces the effective latency of accessing memory in both single-core and multi-core systems. To accomplish this, the dissertation shows that all last level cache misses can be separated into two categories: dependent cache misses and independent cache misses. Independent cache misses have all of the source data that is required to generate the address of the memory access available on-chip, while dependent cache misses depend on data that is located off-chip. This dissertation proposes that dependent cache misses are accelerated by migrat…
Hitherto discovered approaches analyze the execution time of a real time application on all the possible cache hierarchy setups to find the application specific optimal two level inclusive data cache hierarchy to reduce cost, space and energy consumption while satisfying the time deadline in real time Multiprocessor Systems on Chip. These brute force like approaches can take years to complete. Alternatively, memory access trace driven crude estimation methods can find a cache hierarchy quickly by compromising the accuracy of results. In this article, for the first time, we propose a fast and a…
Conventional cache models are not suited for real-time parallel processing because tasks may flush each other's data out of the cache in an unpredictable manner. In this way the system is not compositional so the overall performance is difficult to predict and the integration of new tasks expensive. This paper proposes a new method that imposes compositionality to the system?s performance and makes different memory hierarchy optimizations possible for multimedia communicating tasks when running on embedded multiprocessor architectures. The method is based on a cache allocation strategy that as…
In this work, we propose FUSE, a novel GPU cache system that integrates spin-transfer torque magnetic random-access memory (STT-MRAM) into the on-chip L1D cache. FUSE can minimize the number of outgoing memory accesses over the interconnection network of GPU's multiprocessors, which in turn can considerably improve the level of massive computing parallelism in GPUs. Specifically, FUSE predicts a read-level of GPU memory accesses by extracting GPU runtime information and places write-once-read-multiple (WORM) data blocks into the STT-MRAM, while accommodating write-multiple data blocks over a s…
Cache coherence protocols such as MESI that use writer-initiated invalidation have high complexity and sometimes have poor performance and energy usage, especially under false sharing. Such protocols require numerous transient states, a shared directory, and support for core-to-core communication, while also suffering under false sharing. An alternative to MESI's writer-initiated invalidation is self-invalidation, which achieves lower complexity than MESI but adds high performance costs or relies on programmer annotations or specific data access patterns. This paper presents Neat, a low-comple…
A modern GPU aims to simultaneously execute more warps for higher Thread-Level Parallelism (TLP) and performance. When generating many memory requests, however, warps contend for limited cache space and thrash cache, which in turn severely degrades performance. To reduce such cache thrashing, we may adopt cache locality-aware warp scheduling which gives higher execution priority to warps with higher potential of data locality. However, we observe that warps with high potential of data locality often incurs far more cache thrashing or interference than warps with low potential of data locality.…
As the number of cores in a single chip increases, a typical implementation of coherence protocol adds significant hardware and complexity overhead. Besides, the performance of CMP system depends on the data access latency, which is highly affected by coherence protocol and on-chip interconnect. In this paper, we propose PPB (Phase-Priority Based) cache coherence protocol, an optimization of modern directory coherence protocol. We take advantage of the observation that transient states occur in directory coherence protocol, resulting in some unnecessary transient states and stalling. PPB cache…
It is generally observed that the fraction of live lines in shared last-level caches (SLLC) is very small for chip multiprocessors (CMPs). This can be tackled using promotion-based replacement policies like re-reference interval prediction (RRIP) instead of LRU, dead-block predictors, or reuse-based cache allocation schemes. In GPU systems, similar LLC issues are alleviated using various cache bypassing techniques. These issues are worsened in heterogeneous CPU-GPU systems because the two processors have different data access patterns and frequencies. GPUs generally work on streaming data, but…
As industry moves toward chiplet-based designs, the insertion of hardware Trojans poses a significant threat to the security of these systems. These systems rely heavily on cache coherence for coherent data communication, making coherence an attractive target. Critically, unlike prior work, which focuses only on malicious packet modifications, a Trojan attack that exploits coherence can modify data in memory that was never touched and is not owned by the chiplet which contains the Trojan. Further, the Trojan need not even be physically between the victim and the memory controller to attack the…
Utilizing on-chip caches in embedded multiprocessor-system-on-a-chip (MPSoC) based systems is critical from both performance and power perspectives. While most of the prior work that targets at optimizing cache behavior are performed at hardware and compilation levels, operating system (OS) can also play major role as it sees the global access pattern information across applications. This paper proposes a cache-conscious OS process scheduling strategy based on data reuse. The proposed scheduler implements two complementary approaches. First, the processes that do not share any data between the…
Persistent memory (pmem) products bring the persistence domain up to the memory level. Intel recently introduced the eADR feature that guarantees to flush data buffered in CPU cache to pmem on a power outage, thereby making the CPU cache a transient persistence domain. Researchers have explored how to enable the atomic durability for applications' in-pmem data. In this paper, we exploit the eADR-supported CPU cache to do so. A modified cache line, until written back to pmem, is a natural redo log copy of the in-pmem data. However, a write-back due to cache replacement or eADR on a crash overwr…
The proliferation of heterogeneous chip multiprocessors in recent years has reached unprecedented levels. Traditional homogeneous platforms have shown fundamental limitations when it comes to enabling high-performance yet-ultra-low-power computing, in particular in application domains with real-time execution deadlines or criticality constraints. By combining the right set of general purpose cores and hardware accelerators together, along with proper chip interconnects and memory technology, heterogeneous chip multiprocessors have become an effective high-performance and low-power computing al…
Future multiprocessor chips will integrate many different units, each tailored to a specific computation. When designing such a system, the chip architect must decide how to distribute limited system resources such as area, power, and energy among the computational units. We extend MultiAmdahl, an analytical optimization technique for resource allocation in heterogeneous architectures, for energy optimality under a variety of constant system power scenarios. We conclude that reduction in constant system power should be met by reallocating resources from general-purpose computing to heterogeneo…
Current day processors employ multi-level cache hierarchy with one or two levels of private caches and a shared last-level cache (LLC). An efficient cache replacement policy at LLC is essential for reducing the off-chip memory transfer as well as conflict for memory bandwidth. Cache replacement techniques for inclusive LLCs may not be efficient for multilevel cache as it can be shared by enormous applications with varying access behavior, running simultaneously. One application may dominate another by flooding of cache requests and evicting the useful data of the other application. From the pe…
In a multiprocessor system on chip (MPSoC) IC the processor is one of the highest heat dissipating devices. The temperature generated in an IC may vary with floor plan of the chip. This paper proposes an integration and thermal analysis methodology to extract the peak temperature and temperature distribution of 2-dimensional and 3-dimensional multiprocessor system-on-chip. As we know the peak temperature of chip increases in 3-dimensional structures compared to 2-dimensional ones due to the reduced space in intra-layer and inter-layer components. In sub-nanometre scale technologies, it is inev…
Network-on-Chip (NoC) has become a popular choice for connecting a large number of processing cores in chip multiprocessor design. In a conventional NoC design, most of the area in the router is occupied by the buffers and the crossbar switch. These two components also consume the majority of the router's power. Much of the research in NoC has been based on the conventional router microarchitecture. We propose a novel router microarchitecture that treats the router itself as a small network of the ring topology. It eliminates the large crossbar switch in the conventional design. In addition, n…
In recent years, researchers have explored use of non-volatile devices such as STT-RAM (spin torque transfer RAM) for designing on-chip caches, since they provide high density and consume low leakage power. A common limitation of all non-volatile devices is their limited write endurance. Further, since existing cache management policies are write-variation unaware, excessive writes to a few blocks may lead to a quick failure of the whole cache. We propose an architectural technique for wear-leveling of non-volatile last level caches (LLCs). Our technique uses cache-coloring approach which adds…
Processing-in-memory (PIM) architectures have seen an increase in popularity recently, as the high internal bandwidth available within 3D-stacked memory provides greater incentive to move some computation into the logic layer of the memory. To maintain program correctness, the portions of a program that are executed in memory must remain coherent with the portions of the program that continue to execute within the processor. Unfortunately, PIM architectures cannot use traditional approaches to cache coherence due to the high off-chip traffic consumed by coherence messages, which, as we illustr…
In this paper, we propose a 'full-stack' solution to designing high capacity and low latency on-chip cache hierarchies by starting at the circuit level of the hardware design stack. First, we propose a novel Gain Cell (GC) design using FDSOI. The GC has several desirable characteristics, including ~50% higher storage density and ~50% lower dynamic energy as compared to the traditional 6T SRAM, even after accounting for peripheral circuit overheads. We also exploit back-gate bias to increase retention time to 1.12 ms (~60x of eDRAM) which, combined with optimizations like staggered refresh, mak…
Network-on-Chip (NoC) is currently the paradigm of choice to interconnect the different components of System-on-Chips (SoCs) or Chip Multiprocessors (CMPs). As the levels of integration continue to grow, however, current NoCs face significant scalability limitations and have prompted research in novel interconnect technologies. Among these, wireless intra-chip communications have been under intense scrutiny due to their low latency broadcast and architectural flexibility. Thus far, the practicality of the idea has been studied from the RF front-end and the network interface perspectives, where…
Directory-based protocols have been the de facto solution for maintaining cache coherence in shared-memory parallel systems comprising multi/many cores, where each store instruction is eagerly made globally visible by invalidating the private cache (PC) backups of other cores. Consequently, the directory not only consumes large chip area, but also incurs considerable energy consumption and performance degradation, due to the large number of Invalidation/Ack messages transferred in the interconnection network and resulting network congestion. In this paper, we reveal the interesting fact that t…
Last level caches (LLCs) occupy a large chip-area and there size is expected to grow further to offset the limitations of memory bandwidth and speed. Due to high leakage consumption of SRAM device, caches designed with SRAM consume large amount of energy. To address this, use of emerging technologies such as spin torque transfer RAM (STT-RAM) has been investigated which have lower leakage power dissipation. However, the high write latency and power of it may lead to large energy consumption which present challenges in its use. In this report, we propose a cache reconfiguration based technique …
Electromagnetic fault injection (EMFI) is a well known technique used to disturb the behaviour of a chip for weakening its security. These attacks are mostly done on simple microcontrollers. On these targets, the fault effects are relatively simple and understood. Exploiting EMFI on modern system-on-chips (SoCs), the fast and complex chips ubiquitous today, requires to understand the impact of such faults. In this paper, we propose an experimental setup and a forensic process to create exploitable faults and assess their impact on the SoC micro-architecture. On our targeted SoC (a BCM2837), th…
There has been a significant increase in leakage energy dissipation of CMOS circuits with each technology generation. Further, due to their large size, last level caches (LLCs) spend a large fraction of their energy in the form of leakage energy and hence, addressing this has become extremely important to meet the challenges of chip power budget. For addressing this, several techniques have been proposed. However, most of these techniques require offline profiling and hence cannot be used for real-life systems which usually run multitasking programs, with possible pre-emptions. In this paper, …
One of the most critical aspects of integrating loosely-coupled accelerators in heterogeneous SoC architectures is orchestrating their interactions with the memory hierarchy, especially in terms of navigating the various cache-coherence options: from accelerators accessing off-chip memory directly, bypassing the cache hierarchy, to accelerators having their own private cache. By running real-size applications on FPGA-based prototypes of many-accelerator multi-core SoCs, we show that the best cache-coherence mode for a given accelerator varies at runtime, depending on the accelerator's characte…
In this paper, we investigate the impact of T_{ox} and Vth on power performance trade-offs for on-chip caches. We start by examining the optimization of the various components of a single level cache and then extend this to two level cache systems. In addition to leakage, our studies also account for the dynamic power expanded as a result of cache misses. Our results show that one can often reduce overall power by increasing the size of the L2 cache if we only allow one pair of Vth/T_{ox} in L2. However, if we allow the memory cells and the peripherals to have their own Vth's and T_{ox}'s, we …
Modern multicore processors are employing large last-level caches, for example Intel's E7-8800 processor uses 24MB L3 cache. Further, with each CMOS technology generation, leakage energy has been dramatically increasing and hence, leakage energy is expected to become a major source of energy dissipation, especially in last-level caches (LLCs). The conventional schemes of cache energy saving either aim at saving dynamic energy or are based on properties specific to first-level caches, and thus these schemes have limited utility for last-level caches. Further, several other techniques require of…
On-chip communication infrastructure is a central component of modern systems-on-chip (SoCs), and it continues to gain importance as the number of cores, the heterogeneity of components, and the on-chip and off-chip bandwidth continue to grow. Decades of research on on-chip networks enabled cache-coherent shared-memory multiprocessors. However, communication fabrics that meet the needs of heterogeneous many-cores and accelerator-rich SoCs, which are not, or only partially, coherent, are a much less mature research area. In this work, we present a modular, topology-agnostic, high-performance on…
In this thesis, we describe a new, practical approach to integrating hardware-based data compression within the memory hierarchy, including on-chip caches, main memory, and both on-chip and off-chip interconnects. This new approach is fast, simple, and effective in saving storage space. A key insight in our approach is that access time (including decompression latency) is critical in modern memory hierarchies. By combining inexpensive hardware support with modest OS support, our holistic approach to compression achieves substantial improvements in performance and energy efficiency across the m…
Spin-Transfer Torque RAM (STTRAM) is a promising alternative to SRAM in on-chip caches due to several advantages. These advantages include non-volatility, low leakage, high integration density, and CMOS compatibility. Prior studies have shown that relaxing and adapting the STTRAM retention time to runtime application needs can substantially reduce overall cache energy without significant latency overheads, due to the lower STTRAM write energy and latency in shorter retention times. In this paper, as a first step towards efficient prefetching across the STTRAM cache hierarchy, we study prefetch…
General trends in computer architecture are shifting more towards parallelism. Multicore architectures have proven to be a major step in processor evolution. With the advancement in multicore architecture, researchers are focusing on finding different solutions to fully utilize the power of multiple cores. With an ever-increasing number of cores on a chip, the role of cache memory has become pivotal. An ideal memory configuration should be both large and fast, however, in fact, system architects have to strike a balance between the size and access time of the memory hierarchy. It is important …
