原创 关于FPGA实现PCI initiator的问题

about pci32_v4_11 core's initiator question
Set core to initiator mode and start state-mechine,but the DEVSEL_N,TRDY_N signals inactive always.
1.PC open device;
2.Set PCI command's 0x0007;
3.reset the initiator controler;
4.PC request a memory space(a array) and send the address to FPGA's Target_Addr;
5.the burst lenth is the constant value in code;
6.PC start state-mechine for initiator write;
7.the AD[31:0],CBE[3:0],FRAME_N,IRDY_N,REQ_N signals is phase normal,but the GNT_N,DEVSEL_N,TRDY_N signals phase error.
8.GNT_N is inserted when REQ_N active,but the GNT_N keep low-level whenever REQ_N inactive.
9.the DEVSEL_N,TRDY_N signals keep inactive all the time at initator state.
10.modify code in XPCI_USER file:
always @(posedge CLK or posedge RST)
begin : edge_detect
 if (RST) M_DATAQ = 1'b0;
 else M_DATAQ = M_DATA;
end
assign M_DATA_FELL = !M_DATA & M_DATAQ; //falling-edge detect
always @(posedge CLK or posedge Rst_Initor_St)
begin : watch_statu
 if (Rst_Initor_St)
  begin
   FATAL <= 1'b0;
   RETRY <= 1'b0;
  end
 else if (!M_ADDR_N)// clear at beginning
  begin
   FATAL <= 1'b0;
   RETRY <= 1'b0;
  end
 else if (M_DATA)// latch until end
  begin
   FATAL <= CSR[39] | CSR[38];
   RETRY <= CSR[36];
  end
end
always @(posedge CLK or posedge Rst_Initor_St) begin : initiator_fsm
 if ( Rst_Initor_St ) begin
  STATE <= IDLE_S;
  FatalErr <= 1'b0;
 end else case (STATE)
  IDLE_S :begin
   if ( START_FG ) STATE <= REQ_S;
   else STATE <= IDLE_S;
  end  
  REQ_S : STATE <= WRITE_S;   
  WRITE_S : if (M_DATA_FELL) begin 
    if (FATAL)
     STATE <= DEAD_S;
    else if (RETRY)
     STATE <= RETRY_S;
    else
     STATE <= DONE_S;
   end   
  RETRY_S : STATE <= REQ_S;   
  DONE_S : STATE <= IDLE_S;   
  DEAD_S : begin
   STATE <= DEAD_S;
   FatalErr <= 1'b1;
  end
  default : STATE <= IDLE_S;
  endcase
end
always @(posedge CLK or posedge RST)
begin
 if (RST) begin
    M_READY <= 1'b0;
  M_WRDN  <= 1'b0;
 end else begin
    M_READY <= SynSTART;
  M_WRDN  <= START;
   end
end
always @(posedge CLK or posedge Rst_Initor_St)
begin : transfer_counter
 if (Rst_Initor_St) XFER_CNT <= 9'h0;
 else if (START_FG) XFER_CNT <= 9'h00F; //burst length
 else if ( Fifo_RdReq )XFER_CNT <= XFER_CNT - 1;
end
always @(posedge CLK or posedge Rst_Initor_St)
begin : driving_complete
 if (Rst_Initor_St)  COMPLETE_reg <= 1'b0;
 else if (START_FG) COMPLETE_reg <= 1'b0;
  else if(XFER_CNT == 9'h1) COMPLETE_reg <= 1'b1;
end
assign REQUEST = (STATE == REQ_S);
assign REQUESTHOLD = 1'b0;
assign COMPLETE    = COMPLETE_reg;
assign bar0_dout   = M_ADDR_N ? Fifo_Out : Target_Addr;
assign M_CBE       = M_ADDR_N ? 4'b0000  : 4'b0111;
assign Fifo_RdReq  = M_SRC_EN & M_DATA;

上图中，没有观察到DEVSEL#,TRDY#信号的原因是chipscop的观察点选错了。应该选devsel_d和trdy_d。

上图是burst长度为32个DWORD的波形，目的地址为0xFF7FF004。上图中可看出，FPGA已经发送了32个DWORD，收到3次从设备的STOP信号，故Retry了3次；
目的地址的说明：0xFF7FF004地址是用windriver软件查询到挂到南桥上的一个PCI设备，该设备为inter pro/VM 网卡设备，VID=8086,DID=27DC。该设备用windriver可查到有两个空间，一个是I/O空间，一个是存储器空间。存储空间的地址范围为:0xFF7FF000~0Xff7FFFFF.
用windriver对存储空间偏移为0X04的地址访问(物理地址: 0xFF7FF004)，发现该地址可读可写。
最新测试步骤：
1、 用windrver对0xFF7FF004地址，写为全0；
2、 启动FPGA的burst传输，如上图，收到GNT#,TRDY,DEVSEL#信号，FPGA开始传输数据；
3、 收到总线上的STOP#信号，FPGA待SOTP#消失后继续传输；
4、 待数据传输完成后，用windriver去读地址: 0xFF7FF004，数据为全0。即数据根本没有写到目标设备。

疑问：
1、 上述的目标设备地址: 0xFF7FF004给法是否正确？
2、 能否在windows下申请一个物理地址，然后作为FPGA要传输的目的地址？如何申请，用什么工具可直接查看windows的物理内存数据？
3、 从设备发了3次sotp#信号过来，是否属正常现象？
4、 STOP#信号消失后，FPGA进行retry传输。这时Burst的目的地址是否还是0xFF7FF004；
5、 在windriver下可对0xFF7FF004地址进行可读可写，但为何用FPGA不能写入。从设备还要重新打开一次？

用windriver测试，把FPGA的burst目的地址设为内存的物理地址:0x3F749F00，然后FPGA作为inititator发起数据传输，观察物理内存中的数据:
        地址：0x3F749F00，数据：0x0000009C
                    0x3F749F04，数据：0x00000093
                    0x3F749F0C，数据：0x00000095
                    0x3F749F10，数据：0x00000096
                    0x3F749F14，数据：0x00000097
                    0x3F749F18，数据：0x00000098
                    0x3F749F1C，数据：0x00000099
地址0x3F749F20后的数据已经杂乱无章了；
用chipscop观察FPGA发送的数据：
FPGA收到STOP#信号3次，在收到第一次sotp#信号之前我们把他叫做是发送的0次数据，第二次stop之前(第一次stop之后)我们把他叫做第二次数据，依次类推。。。
0次数据：0x00000007E, 0x00000007F,0x000000080, 0x000000081, 0x000000082, 0x000000083, 0x000000084, 0x000000085, 0x000000086, 0x000000087
1次数据：0x000000088, 0x000000089, 0x00000008A, 0x00000008B, 0x00000008C, 0x00000008D, 0x00000008E, 0x00000008F, 0x000000090, 0x000000091
2次数据：0x000000092, 0x000000093, 0x000000094, 0x000000095, 0x000000096, 0x000000097, 0x000000098, 0x000000099, 0x00000009A, 0x00000009B
3次数据：0x00000009C, 0x00000009D

疑问：FPGA在发起这几次数据传输时都是给的同一个地址：0x3F749F00，而MEM中，貌似只收到7个正确的数据，这几个正确的数据不在FPGA发起数据头尾？

  对于昨天2010-7-21-2的问题是由于每次burst的目的地址没有自动增加造成的。最新测试数据如下(MEM的地址从0x3F749F00开始的32个DWORD)：从以上的数据可看出：MEM中的数据呈现有规律的错误，而FPGA发送的数据为连续数据。
错误原因：FPGA在user端没有正确的给定数据；下面我们来观察chipscop的波形：

上图中，收到从设备的3次sotp#信号，而实际传输了4次，每次都传输8个DWOD，共传输了32个DWORD。第一次Frame有效时，给出的是0x0x3F749F00地址，第二次Frame有效时，给出的是0x3F749F20地址，第三次Frame有效时，给出的是0x3F749F40地址，第二次Frame有效时，给出的是0x3F749F60地址.

我们现在来观察一次连续8个DWORD的传输情况：

注意：上图中，trdyq_n,deselq_n,stopq_n比真正的PCI总线上的时序要延时一个时钟周期。因为这里是扑捉的其锁存信号。
观察M_DATA_VLD信号，总共占用8个时钟周期的宽度，即PCI总线上实际传输了8个DWORD。观察M_SRC_EN信号，总共占用10个时钟周期的宽度，若使用M_SRC_EN信号作为用户应用的FIFO读使能信号则会发生错误。
现在我们来观察PCI的总线时序：上图中，Frame#信号一共有效11个时钟周期，我们就从1开始编号。
Fram#有效1周期:这时，ADO为正确的地址信号，CBO表示正确命令信号，这时对方TRDY#信号还没有给出，但M_SRC_EN信号已经有效了；
Fram#有效2周期:对方的TRDY#信号无效，故为等待周期，但M_SRC_EN信号也仍然有效；
Fram#有效3周期:TRDY#有效(trdyq为TRDY#的一个锁存延时周期)，DEVSEL#有效(deselq为DEVSEL#的一个延时周期)，即数据开始传输第一个DWORD；该周期M_SRC_EN无效，前两个时钟周期M_SRC_EN都有效，故肯定丢失了一个DWORD数据；
。。。。。。

从以上看出，M_SRC_EN不仅时序上不满足要求，而且有效的周期数也不能满足要求；

解决办法：

Using M_SRC_EN to present data for the next data phase may require additional control logic depending on the type of data source present in the user application. Keep in mind that the M_SRC_EN signal advances data pointers (and possibly the initiator address pointer) in anticipation of the next data phase, which may or may not complete with successful data transfer.
If a pointer is advanced, and the data is never transferred, then the user application must decide what to do with the non-transferred data. In the case of prefetchable data sources, such as RAM or a register file, the data can be discarded. The original data remains in the RAM or the register file for future use.
This also applies in cases where a FIFO is used as a rate matching buffer and the contents of the FIFO are flushed after a transaction. Any non-transferred data is discarded from the FIFO, but the original data still remains in the source that originally provided it.
For non-prefetchable data sources, as is the case when a FIFO is the data source, pulling data out of the FIFO may be destructive. The unused data must be restored in the data source so it is available for future use should it not be transferred. This may require decrementing internal counters or keeping a shadow copy of the previous data values.
Conditions requiring a backup may arise at the end of an initiator write transfer where the target signals some form of disconnect, terminating the transaction before the initiator is able to complete the full transfer. In these cases, the user application is not immediately aware of the termination condition, and will have advanced the data source too many
times. This condition can also arise during data transfers that span multiple bus transactions.
One way to determine the number of times the data pointer (and possibly the initiator address pointer) has been over-advanced during a burst write is to monitor the difference in the number of cycles M_SRC_EN and M_DATA_VLD have asserted during a transaction. During initiator writes, the signal M_DATA_VLD represents the number of data phases that actually complete with data transfer

最终测试结果：FPAG连续传输数据给计算机内存，速度为14532Byte/248us => 58.597MByte/s