My visual force page field looks like this, as they are displayed in pageblocktable:




You must be aware that for lookup fields the ID's are stored in the field named ending with '_lkid'.
In my case for 'AllocatedTo' it is 'AlocatedTo_lkid'.
So on click of 'parent inventory' lookup, I fetch the user id from 'AlocatedTo_lkid' field and pass it to my page for filteration.

On the above page I have overriden the salesforce openlookup() method and displayed my visualforce page for both lookups, To display the list.

NOTE: If you override the openLookup() method you need to create visualforce lookup page for all the lookup fields.

my openlookup method:

function openLookup(baseURL, width, modified, searchParam) {
       width = width+100;
       var lookupType = baseURL.substr(baseURL.length - 3, 3);
       var lookupTypeUser = baseURL.substr(baseURL.length - 18, 18);
       var isCustomLookup = false;
      
  if (lookupType == "005" || lookupTypeUser == "StandardUserLookup") { // lookup for standard user object

           txtId = textFildId(baseURL)

           baseURL = "/apex/CustomUserLookup?txt=" + txtId;

           baseURL = baseURL + "&frm=" + escapeUTF("{!$Component.myForm}");
           if (modified == '1') {
               baseURL = baseURL + "&lksearch=" + searchParam;
           }

           if (txtId.indexOf('AlocatedTo') > -1) {
               isCustomLookup = true;
           }
          
        } else if (isInventoryObj(baseURL)) {     //isInventoryObt(): Checked if lookup is related to my object
           txtId = textFildId(baseURL)
           var fieldLocation = txtId
           var allocatedfldID = decodeURIComponent(fieldLocation.replace('Parent','AlocatedTo_lkid'));
           // fieldLocation also contains the row number. So you can get corresponding ID
           var selectedUserId = document.getElementById(allocatedfldID).value;  
           baseURL = "/apex/FilteredInventoryLookup?txt=" + txtId+"&userId="+selectedUserId;          
           baseURL = baseURL + "&frm=" + escapeUTF("{!$Component.myForm}");
           if (modified == '1') {
               baseURL = baseURL + "&lksearch=" + searchParam;
           }

           if (txtId.indexOf('Parent') > -1) {
               isCustomLookup = true;
           }

       }
       console.log(isCustomLookup);


       if (isCustomLookup == true) {
           openPopup(baseURL, "Popup", 350, 550, "width=" + width + ",height=480,toolbar=no,status=no,directories=no,menubar=no,resizable=yes,scrollable=no", true);
       }
   
}

__________________________________________________________________________

On the visualforce lookup page, you need to call salesforce javascriptmethod named : lookupPick2() on the outputlink, as follows:

<apex:pageBlockTable value="{!inventoryList }" var="inventory" id="tblResult">
           <apex:column >
                  <apex:facet name="header">
                               <apex:outputPanel >Name</apex:outputPanel>
                                    </apex:facet>
                                        <apex:outputLink value="javascript:top.window.opener.lookupPick2('{!FormTag}','{!TextBox}_lkid','{!TextBox}','{!inventory.Id}','{!inventory.Name}', false)" rendered="{!NOT(ISNULL(inventory.Id))}">{!inventory.Name}</apex:outputLink>
                                    </apex:column>
</apex:pageBlockTable>

______________________________________________________________________________

in the lookup controller, have following methods:
    public performSearch() {
         inventoryList = [select...........];
    }
   
    /* used by the visualforce page to send the link to the right dom element   */
    public string getFormTag() {
        return System.currentPageReference().getParameters().get('frm');
    }

   /* used by the visualforce page to send the link to the right dom element for the text box   */
    public string getTextBox() {
        return System.currentPageReference().getParameters().get('txt');
    }




Thanks,
Satyan