PCI Express (2002) (3GIO)
  (PCI/PCIe - is independent bus - also used in non Intel systems)

    Conventional PCI
      32-bit data bus / 33 MHz
        64-bit/66MHz available but seldom implemented.
      4-bit C/BE - provides transaction type or mask of data bytes
      Frame - signals a transaction consisting
        Single address/command transfer
        0-many word transfers.
      4 arbitration pairs
      4 interrupt lines. 
        Or uses Message Signaled Interrupts, MSI, over data lines.
      Single ended signaling.
      Reflected wave.
      256 buses/32 devices/8 functions
      Limit of 4 devices per bus.

      ~ 133MB/s transfer speed
 
  PCIe builds on top of the conventional PCI controller/protocol

  However, it completely scraps the hardware interface,
    Replacing it with 1 or more pairs of point to point serial lanes.

    Each device/controller interface assigned its own interface and bus ID.

  PCIe is a networked bus.
    Point to point connections between controller and target 
    or routing switch residing on a device.

  PCIe packetizes PCI transactions 
    serializes all information transmitted
      Address, data, and control signals all transmitted over serial link.

    Uses packet encapsulated data transfers similar to networking.

    Controller also uses switching hub to interleave various transactions.
 
    Because communication is packetized, as long as different transactions 
      don't involve same endpoints, 
      multiple task communication can be interleaved.

    Packet up to 64 bytes long (Windows).
      * other sizes 128, 256, 512, 1024, 2048, 4096
        Requires support of BIOS
        Device initiation will communicate maximum supported.
    
    Information being transmitted is first packetized. 
    

    

    
Data Link
Layer Frame
    

      

      

        
Transaction
Start Frame
Seq#
header
        
data
 
CRC
End Frame
      
      
     
     
Data Link
Layer End
     

    
        
    Packets serialized.

    8,10b encoded to eliminate long strings of zeros or 1s.
      RLL helps with clock recovery when sender and receiver have dependable
        reference clocks.
 
      But a reference clock signal is available for more dodgy situations.

    Converted to differential signal, transmitted, and converted back.
      Still uses NRZ style.

    # The overhead, non-data bits, fairly high, but transmission speed
    #  in multi-Gigabit/sec. range.

    
    Differential Signaling

   http://xillybus.com/tutorials/pci-express-tlp-pcie-primer-tutorial-guide-1

  A pair of differential lines (2 pair of lines) 
    carrying data in each direction is a lane.
    
  Each connection between controller and device 
    consisting of 1 or more lanes is a link.

  PCIe is capable of full duplex.

  For devices needing higher bandwidth, 
    More than one lane may be used.
      Bytes sequentially interleaved (striped) between lanes, 
      so order of bytes known even if they don't arrive at exact same time. 
        Byte sized distribution limits skew issues.

    Up to 32 lanes available but 16 usually largest in most consumer systems.
      1,2,4,8,16,32

    Multi-lane more commonly used with video cards and RAID controllers. 
 
    Small blocks of data see little speed improvement on multi-lane because
      current packet must be completed before next sent.
 
    Also, if sender or receiver slow at processing data,  
      single lane sufficient.

  Switching module ties each of the "lanes" together into the interface.

  Other lines :
    Several +3 volt and +12 volt lines, Ground.

    Differential reference clock and clock request lines.
      #optional, PCIe uses RLL encoding. 
  
    PRSNT1,2 -  used to detect hot-plugged card.

    Various lines for resetting and testing of interface card.

  Connectors - vary according to number of lanes, 
    Slots can vary in length.

    

    Wide slots may or may not actually have more lanes.

    Shorter slots can sometimes have open ends to take longer cards.
     
    Card with fewer lanes can use wider slot.

    Wider card in narrow slot should detect and adjust for it.
      Loss of throughput.

    Number of lanes used - recognized/negotiated electronically.
      8 lane slot wired for only 2 will still work, just slower.

    Controllers monitor lane behavior and are able to ignore poorly behaving
      lanes (if additional lanes available).

  PCIe 1a (2003) -  250MB/s per lane throughput. 
    * 2.5 GT/sec. per lane - giga-transfers.
    * Uses 8/10b - 80% actual data.
    * Single lane - mechanical hard drives < 130 MB/s
    * multiple lanes provide throughput for SSD and video.
 
  PCIe 2.0 (2007) 500MB/s per lane. x32 ~ 16GB/s total.  
    5 GT/s per lane.
    * Uses 8/10b 
    * SATA 3 - 2-6 Gbits/s  

  PCIe 3.0 (2010) 984.6 MB/s per lane.
    8 GT/s per lane.
    * Uses 128/130b, 98% data. 
    * Improved control, error correction, etc.
    Also uses scrambling to improve correct transmission and limit EMF. 
    
  PCIe 4.0 (2017) 1969 MB/s per lane..
    16.0 GT/s per lane.
    * Uses 128/130b
 
  PCIe 5.0 (tentative Q2 2019) 3938 MB/s per lane.
    32.0 GT/s per lane.
    * Uses 128/130b


Next