PCI Express (2002) (3GIO)
  (PCI/PCIe - is independent bus - also used in non Intel systems)


  PCI-E controller is a hub with individual point to point connections to 
    each device.

    Assigns each 'device' connection its own bus ID.
      Treats each connection as separate bus which just happens to have a 
      single device on it.

  Compatible at device, driver, and system level with standard PCI. 
    Serial transfer circuitry new.

    Once a task starts the bus is occupied until task completes,
      is suspended, or aborts.

    On PCIe, because task communication is packetized, as long as task don't
      involve same endpoint, multiple task communication can be interleaved.

  PCI-E allows device interfaces, SATA, PATA, Video cards. 

  Implemented on copper, fiber, or others.

  Copper implementation includes a pair of lines used for clock signal.

  Data transmitted serially in paired lines using differential signaling.
    Differential signaling allows externally induced errors to cancel out.
    

    
    Differential Signallng

    Information being transmitted is first packetized. 
    

    

    
Data Link
Layer Frame
    

      

      

        
Transaction
Start Frame
Seq#
header
        
data
 
CRC
End Frame
      
      
     
     
Data Link
Layer End
     
    
        
   https://xillybus.com/tutorials/pci-express-tlp-pcie-primer-tutorial-guide-1

    Remember data is coming from system/PCI bus (NRZ).

    Data packetized and then serialized (still NRZ).
      Packet up to 64 bytes long.
    
    8,10b encoded to elimiate long strings of zeros or 1s.

    Converted to differential signal, transmitted, and converted back.

    # The overhead, non-data bits, fairly high, but transmission speed
    #  in multi-Gigibit/sec. range.

  Two pairs of these lines (one pair for each direction) carrying data in 
    opposite directions make up a "lane".

  Data lanes used to move address, data, and control signals.

  Additional independent lanes can be added. Up to 32 lanes.
 
  For devices needing higher bandwidth, 
    More than one lane used, but each lane functions independently.
      Individual byte serially sent over single lane.
      Bytes interleaved between lanes, so order of bytes known even if
      they don't arrive at same time. (No skew issue). 

    Up to 32 lanes available but 16 usually largest in most consumer systems.
      1,2,4,8,16,32

    Multi-lane more commonly used with video cards and RAID controllers. 
 
  Switching module ties each of the "lanes" together into the interface.

  If multiple lanes exist, data bytes interleaved (striped) between lanes. 
    Interface circuitry must be able to rebuild byte order (similar to how
      packets are sent across the Internet).

    Small blocks of data see little speed improvement on multi-lane because
      current packet must be completed before next sent.
 
    Also, if sender or receiver slow at processing data,  
      single lane sufficient.

  Connectors - vary according to number of lanes, 
    Slots can vary in length.

    Wide slots may or may not actually have more lanes.

    Shorter slots can sometimes have open ends to take longer cards.
     
    Card with fewer lanes can use wider slot.

    Number of lanes used - recognized/negotiated electronically.
      8 lane slot wired for only 2 will still work, just slower.

    Controllers monitor lane behavior and are able to ignore poorly behaving
      lanes (if additional lanes available).

  PCIe 1a (2003) -  250MB/s per lane. 
    * 2.525 Gb/sec. per lane pair (2.5 GT/s) giga-transfers.
    * Uses 8/10b 
    * ~ 250MB/sec. data. 
    * Single lane - mechanical hard drives <100 MB/s
    * multiple lanes provide throughput for SSD and video.
 
  PCIe 2.0 (2007) 500MB/s per lane. x32 ~ 16GB/s total.  
    5 Gigabit transfers per second (GT/s) per lane but uses 8b/10b, so only 
      80% actual information.
    * approx 400 MB/s - faster than standard hard drive, slower than new SSDs.
    * SATA 3 - 2-6 Gbits/s 

  PCIe 3.0 (Official Nov. 2010) Improved control, error correction, etc.
    Also uses a form of recoverable scrambling with only 1.5% overhead 
    and 128b/130b RLL. Transfer of 10 GT/s with reduced timing overhead 
    may effectively double current transfer rates.
    
  PCIe 4.0 offically announced June 8, 2017.
    16GT/s - better monitering of lanes and improved dynamic lane assignment.
 
  PCIe 5.0 - tentative release Q2 2019

Next