Jump to content
Sign in to follow this  
RAGE

NEW PAYPAL

Recommended Posts

We advise basing all processing exclusively off of the IPN. IPN is more reliable (doesn't depend on the buyer's browser, retries until successful) and handles e-checks.

 

We advise against processing the Continue button. Processing the Continue button requires a decent amount of technical expertise and knowledge about how PayPal works.

 

We advise against processing Auto-Return in all circumstances as it does not provide proof of payment.


Patrick Breitenbach

Share this post


Link to post
Share on other sites

I've been trying to think some more in terms of the information provided in Patick's above post regarding IPN's and so I'm writing some personal thoughts which may or may not relevantly capture all the aspects involved or indeed available, others may like to clarify or make further suggestions or comments.

 

To me there seems to be a chicken and egg problem occuring pivoting around the customers session.

 

[A]

In my current contribution the customer is still expected to return to the correct site page, namely checkout_process.php, either by clicking continue or being automatically returned. Because if they go to a different site-page, instead of returning to the expected checkout_process.php page, the contents of their cart still remains.

 

The reason why the private IPN script currently does not reset the cart is so that when the customer does return to checkout_process.php osC can then reset the cart and update the session data, i.e remove the checkout data, and redirect the customer to the checkout_sucess.php page as per default osC scenario.

 

However if the IPN script did reset the cart etc then currently upon the customers return osC will automatically redirect them to the checkout_shopping.php page, showing an empty cart.

 

I had started to think that the IPN script could perform it's validation checks from an intermediate set of order tables.

 

[C]

But in order to begin to resolve issue [A] the IPN script still needs to have access to the customers session (which would imply that a pre-order table is not really neccessary). And it may also cause a need for a change to occur in osC as to how it handles a returning customer so that does not occur.

 

It is possible to resolve [C], but I'm not sure whether it is generically possible, other than to make this session check available only to certain 3rd party payment gateway sites, in this case primarily PayPal.

 

But then herein lays the chicken and egg problem is that supposing the customer does return to the site prior to the IPN being received, their session cart info is still needed by the IPN and so the cart (etc) should not be reset.

 

It might be possible to regenerate a new session id and to automatically register some of previous session vars so that the customer is still considered logged in etc.

 

The reason for this is so that the IPN can still pick up the original session irrespective of whether the user has returned to the site. If the user has returned to the site then before a new session_id is created a session var should be set in the original one to signify to the IPN script that it is ok to destroy that session.

 

But supposing that the customer did return to site and was provided with a new session id, they could spoof the system with the old session_id and change the contents of the cart (assuming that the IPN has not been received in the mean time).

This increases the level of verification checks required by the IPN script, which currently because PayPal only allows two options to be associated with a particular product item makes its even more intricate to reconstruct the IPN-cart information entirely.

 

Even though I have not encountered a significant delay (actually it's been more or less instanteous), PayPal makes no guarantee in the timing of sending the IPN, but this might be due to my server environment.

 

We advise against processing the Continue button. Processing the Continue button requires a decent amount of technical expertise and knowledge about how PayPal works.

Is this information covered in the PayPal IPN Manual? It would be appreciate if we could know where to find this information.

 

We advise basing all processing exclusively off of the IPN. IPN is more reliable (doesn't depend on the buyer's browser, retries until successful) and handles e-checks.

Given that PayPal makes no guarantee in the timing of sending an IPN and A, B and C above, is this still entirely feasible?

In regard to e-checks, wouldn't the customer be returned to the site with a pending payment_status? And couldn't the private IPN script process it's update?


"Any fool can know. The point is to understand." -- Albert Einstein

Share this post


Link to post
Share on other sites

I unfortunately do not know exactly how osC works but will try to make a reply.

 

What most carts do is send PayPal a unique order number (best to pass through as "invoice" which PayPal enforces uniqueness on).

 

The shopper pays.

 

PayPal posts an IPN to the store that inludes the order number. The store applies the payment to the order.

 

The shopper returning to the store is a totally independent event and should have no further affect on the processing.

 

If URL-based session are in use, the session ID would need to get passed in the "return" URL. If cookie-based sessions are used then it should be OK. The key is not to do any processing based on the "return" URL.

 

Passing the individual item information is a "nice-to-have" but I usually advise against it because it's more code to watch after and not strictly necessary since all the cart details are available in osC.


Patrick Breitenbach

Share this post


Link to post
Share on other sites
I unfortunately do not know exactly how osC works but will try to make a reply.
The most secure way to access a payment method from osCommerce is to call via secure cURL. This encrypts the transaction. The gateway processes the info on its side and returns an authorization or a decline (possibly with reason). It does this inside the before_process function of the payment method, which is called from the checkout_process page. The checkout process page is called directly from the checkout_confirmation page.

 

Now, PayPal doesn't support this. With PayPal, the process goes slightly differently. From the checkout_confirmation page, the store will post info to PayPal. The customer will verify the info and add new info as necessary. When completed, the customer will use Auto-Return to return to the checkout_process.php page. In that page, the before_process function will check for the transaction authorization (the IPN). If it gets it, it should add the order to the database. If not, then there is a problem. I think that Greg's contribution adds the order to the database anyway, but that is not really correct for how osC works at the moment (although it is consistent with what the check/money order module does--i.e. payment comes after processing). For example, one could spoof this to allow a download of a virtual product without paying.

 

The problem is that currently osC assumes that the payment has been authorized before creating the order. In your paradigm, it needs to create the order and then call PayPal. Unfortunately, that is *not* how osC currently works. It cannot create the order first, as order creation is dependent on payment authorization. AFAIK, this isn't scheduled to be changed until version 3.0 (it's not on the 2.2 workboard).

 

To get back to Greg's question, to work with osCommerce, we need to have the customer auto-return to the site after making their payment. We also need to be able to check for an authorization in the before_process function. Thus, the IPN would have to be available by then. Essentially, the authorization or decline would have to be made at the time of the auto-return (it is checked a trivial amount of time after). If not, then nothing has changed. PayPal still doesn't feed back a usable response to osCommerce. The store owner needs to generate orders based on the PayPal receipts rather than the osCommerce system (otherwise people could order without paying).

 

Hth,

Matt


Always back up before making changes.

Share this post


Link to post
Share on other sites
The problem is that currently osC assumes that the payment has been authorized before creating the order. In your paradigm, it needs to create the order and then call PayPal. Unfortunately, that is *not* how osC currently works. It cannot create the order first, as order creation is dependent on payment authorization. AFAIK, this isn't scheduled to be changed until version 3.0 (it's not on the 2.2 workboard).

the osCommerce PayPal IPN modules takes care of this by using its own checkout_process file (named checkout_paypalipn I think). It creates the order 1st and sets the status to "PayPal Processing". Then sends the customer to paypal to pay. They do not need to click "continue" to return to the store if they do not wish to, as paypal then sends the transaction info to payapl_notify on the store's site and updates to status to pending or processing if it went through. It also stores the transation info in the OSC database with an admin interface, so you can verify the transaction info from inside your stores admin if you wish.

 

To my knowledge the IPN has always functioned like this and the osCommerce paypal IPN module has been around for well over a year.


The only thing necessary for evil to flourish is for good men to do nothing

- Edmund Burke

Share this post


Link to post
Share on other sites

That is correct that PayPal is a hosted processor like AuthorizeNet SIM and PayFlow Link.

 

Is it possible for osC to create the order based exclusively on the IPN? Does the shopper have to be present for osC to complete the order? How does osC store cart contents before they are paid? Couldn't the cart ID be passed to PayPal and PayPal passes it back in the IPN which can be processed to create/update the order?


Patrick Breitenbach

Share this post


Link to post
Share on other sites

It's been awhile since I looked at that contribution and the way that the contribution dreamscape is refering to above works, physically redirects the customer to PayPal via the GET method which allows the customer to change the price etc in the url, PayPal has no way of verifying this change of information and so justs accept the new info for payment.

Also if you download and install that contribution as is, the fact that it stores invalid transactions leaves it open for malicious abuse.

 

To be me Matt seemed to better capture the essence here, I will try and write again maybe, but upon first read I did have the following thoughts in relation to MS2-2.2.

 

[A]

Inline with Matts comments; when the customer does retrun to checkout_process.php osC should have the ability to instigate a curl connection to PayPal and thus confirm payment, this alleviates the issue of whether the IPN has been received in time.

The IPN should still be sent which can consequently generate the order in the event that the customer does not return to the site or if the IPN is received prior to customers return.

 

Just to clarify a little more about what the PayPal_Shopping_Cart_IPN currently does is if the customer clicks the continue button, they are posted back from PayPal to checkout_process.php with the same information as the IPN that PayPal sends. This information is validated (via a call in the before_process) in the same way as the PayPal IPN. If it is verified, then the checkout_process.php is allowed to continue generating the order.

If the customer is Auto-Returned to the site, because there is no PayPal transaction info available, in the before_process function the customers cart is reset and they are redirected to checkout_success.php (i.e. they bypass the order generation of checkout_process).

Currently when they are Auto-Returned in the url there is a parameter signifying this.

So now if the customer returns to the site upon clicking continue button and their IPN information cannot be verified, or, they are not Auto-Returned correctly then they are considered to be spoofing the system and are redirected back to checkout_payment.php.

So now the only way that an order is generated via user interaction is when they return to the site with valid IPN info.

If however the IPN has been received prior to their return, the order is generated, so in the case of when the customer clicks continue that IPN information that they have is disregarded as being a duplicate. And if the customer is Auto-Returned then is everything is seemedly ok - but this is provided that the IPN is received first.

 

I have not had the time to provide a script to prevent downloads from occuring, I did look into this awhile back and came up with an idea based upon registering some new session vars, but that code was not archived properly.

 

In the latter part of Matt's post was the mention of using the auto-redirect, and checking to see if the IPN existed, but to me, this still raises the issue about what to do with the session data because if the IPN has not been received it needs to be stored so that it can be picked up by the IPN and also allows the customers cart to be reset.

 

I think PayPal should seriously consider [A].

 

MS2-2.2 is going to be around for along time, some sites are still using MS1, and the implementations made for MS2-2.2 can be easily adpated into MS2.2-3.


"Any fool can know. The point is to understand." -- Albert Einstein

Share this post


Link to post
Share on other sites

asdasd

Edited by gregbaboolal

"Any fool can know. The point is to understand." -- Albert Einstein

Share this post


Link to post
Share on other sites

My quick responses, maybe others have their opinions.

Is it possible for osC to create the order based exclusively on the IPN?

Because the order is not pre-stored it relys on the session data, so although it can be exclusively created it relys on the existance of correct session information.

 

Does the shopper have to be present for osC to complete the order?

No, the IPN covers this.

 

How does osC store cart contents before they are paid?

In the session data?

 

Couldn't the cart ID be passed to PayPal and PayPal passes it back in the IPN which can be processed to create/update the order?

Well the assumption I personally have currently had here is that this what the session id does, and it is currently passed as the custom variable.


"Any fool can know. The point is to understand." -- Albert Einstein

Share this post


Link to post
Share on other sites

Does osC store cart contents in the database while the shopper is shopping?

 

"I think PayPal should seriously consider [osC should have the ability to instigate a curl connection to PayPal and thus confirm payment]."

 

How would this work exactly? Does it need to be possible to cancel the payment at that point?

 

The best thing I think to do is to totally disregard the "return" and Auto-Return and instead do all of the processing based on the IPN. This makes things very clean and less error-prone. In fact, I would advise setting rm=1 in the PayPal button which makes the "return" be a GET with no IPN data.

 

Also, if cURL isn't already required by osC, I might suggest using fsockopen instead since it's more prevalent. IPN can go over http or https (http is fine since there's no credit card information being transmitted).

Edited by pbreit

Patrick Breitenbach

Share this post


Link to post
Share on other sites

"I think PayPal should seriously consider [osC should have the ability to instigate a curl connection to PayPal and thus confirm payment]."

 

Indirectly I think I was just refering to the RM 2 method of processing the continue button.

 

I would have to look more into the sessions and shopping_basket. I would of and have assumed that when a session is destroyed, i.e it's data removed, then this would occur irrespective of whether it is stored in flat files or the database.

 

At present there seems to be an issue in my contribution with the storing the sessions in the db and the auto-return, becasue when the customer is returned the cart is reset (no order processed) but upon redirect to checkout_success.php the previous session data still remains, this does not seem to occur when clicking the continue button.

 

In regard to connection types the contrib will automaitcally try curl,fsockopen-ssl,fsockopen-http,https and http, in that order.

 

It would now seem that in some way the order must be pre-stored outside of the session. When the customer is returned, they are given a simple thankyou notice, they can then go to their account history to check the status of their order.

Further scripting would be neccessary to enable/disable downloads etc.

There could be problems for people trying to integrate this into existing sites.

And according to Matt this type of functionality is planned for MS3 but MS2- 3 & 4 is still in progress.

The other concern here is when does an order become and order, because if you work directly on the main orders table and have primarily abandoned orders, then the order numbers get messed up. This is something that auctionblox is addressing in the Ebay Auction Manager contrib, with a temporary orders table.


"Any fool can know. The point is to understand." -- Albert Einstein

Share this post


Link to post
Share on other sites

I'm not sure why we keep referencing the 'session' data here. Once the customer completes account registration, there is no longer a 'session'. There is only a customer in the database, and their cooresponding cart contents, that uses a foreign key.

 

The *only* issue with the former IPN module is that malicious customers can chage the query string, and therefore change the total of the purchase.

 

I'm not sure about GET or RM@ or rm=1, but if any of these solve that problem, then that's a step in the right direction.


-------------------------------------------------------------------------------------------------------------------------

NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit.

If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.

Share this post


Link to post
Share on other sites

I know its not the point of the discussion here, and it may well be an intentional design to store invalid transaction, if you look in paypal_notify you'll see the following bit of code, I could not quickly install it this morning to check, but from my test before I could mimick a txn_id that didn't exist, along with the expecting form fields and because the result is invalid and the txn_id does not already exist in the db, reading the code below suggests that it would be stored (to me it just seemed uneccessary).

if ($txn_id && ($response_verified==1 || $response_invalid==1)) {
$txn_check = tep_db_query("select txn_id from " . TABLE_PAYPALIPN_TXN . " where txn_id='$txn_id'");
if (tep_db_num_rows($txn_check)==0) { // If txn no previously registered, we should register it

Chris, it wasn't very cleary to me when you say foriegn key and cart contents, where is the cart contents stored, flat-file, sessions table, or the shopping_basket table?

 

GET or RM@ or rm=1

This is how the customer is returned from the PayPal site


"Any fool can know. The point is to understand." -- Albert Einstein

Share this post


Link to post
Share on other sites

The cart contents is stored in the the customers_basket and customers_basket_attributes table in the database. At the point where the customer completese registration, I feel like the session variable becomes useless. Everythign can be accessed from the DB using the customers_id.


-------------------------------------------------------------------------------------------------------------------------

NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit.

If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.

Share this post


Link to post
Share on other sites

Yeah but when the customer returns to the site the contents in the shopping_basket table is deleted, this occurs in the function reset ( $cart->reset(true) ), which if the IPN has not already been processed should not currently happen.

Edited by gregbaboolal

"Any fool can know. The point is to understand." -- Albert Einstein

Share this post


Link to post
Share on other sites

So, you are saying that when the customer returns, it shows that their cart is still full, becaue if the IPN has not processed, then the cart is not supposed to be emptied.

 

I'm not sure I agree. There is no reason I can think of why the cart shouldn't be emptied on checkout_success, regardless of the startus of the order. Just becaues the cart is emptied, doesnt' mean that the order has been upgraded to Pending by the IPN, and therefore wouldn't be fulfilled until the IPN is processed.


-------------------------------------------------------------------------------------------------------------------------

NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit.

If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.

Share this post


Link to post
Share on other sites

Wizard,

 

Your're seem to be confusing the contib you use, which in the way it currently pre-stores the and redirects the cutomer to PayPal is not good practice.

 

If the order is not pre-stored, then the condrum occurs when the customer returns to the site because the cart should be reset and the session checkout date removed, but because the order is not pre-stored the IPN, for my contrib anyway, relys on the ability to reconstruct the cart and session.

 

At the time of development for my contrib there was no auto-return, and I did not come across anything by PayPal saying that they did not recmmend using RM=2 (they just stated it was available in their Manual - so it was utilized).

 

Looks like pre-storing is the way to go but it has to be effeciently in respect to the naturla flow of the checkout process, i.e no GET redirects. I wanted to work on that contrib but the author wanted to perpetually keep the copyright.


"Any fool can know. The point is to understand." -- Albert Einstein

Share this post


Link to post
Share on other sites

No, I wasn't getting them confused. The IPN contribution I was talking about is the one that has been working great for over a year now.

 

Like I said, the main issue with it is that a malicious user can alter the query string to change the payment amount. IMO, that's where the focus of the discussion should be.

 

Does your contribution not re-direct the customer to PayPal? If so, that would be terrific. I'm not sure if that's the same thing as a GET redirect that you mention.


-------------------------------------------------------------------------------------------------------------------------

NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit.

If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.

Share this post


Link to post
Share on other sites

I strictly followed the methodology used in the original osC PayPal module, so when the customer is click the osC confirmation button they are immediately POSTed to PayPal's site.

 

There is a contribution called held_orders, and also AuctionBlox is using a 'holding' table.

 

I need to look more but I suspect that a pre-storing in any form should occur within the $payment_modules->update_status() function which occurs on the checkout_confirmation.php

 

I will be looking further into Anthony's 'holding' method later.


"Any fool can know. The point is to understand." -- Albert Einstein

Share this post


Link to post
Share on other sites

I think Chris is referring to this contrib:

http://www.oscommerce.com/community/contributions,1352

 

This may be confusing things if that contrib has modified the checkout to store cart contents in the DB (create and order?) before directing the buyer to PayPal to pay.

 

For simplicity's sake, I'd suggest focusing first on a solution that works perfectly for non-eBay sellers of physical goods.

 

Then it would be safer to adapt for digital goods and eBay selling.


Patrick Breitenbach

Share this post


Link to post
Share on other sites

Yes, that is the contribution that currently works.

 

Yes, it adds the paypal order to the database with a special status before the customer is re-directed to PayPal. I still havn't seen a way around that.

 

This was to focus of my point previously, but all of my posts were removed from the thread and deemed 'irrelevent'. All well, I don't really have the energy to post them all again.

 

Suffice it to say, even with the new methods paypal has offered, I still don't see how it can work without posting the order to the database before the customer is re-directed to paypal.


-------------------------------------------------------------------------------------------------------------------------

NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit.

If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.

Share this post


Link to post
Share on other sites

A concern by some people is that storing the order directly into the main orders table causes havoc with the orders number sequence due to abandoned carts, this was expressed to me by a previous storeowner of the contrib you are both refering to.

 

Anthony's 'holding' table would aleviate this (and has other potential benefits not just in the case of IPN), which is I assume is similar to the principle behind held_orders.

Whether the holding table is purely for PayPal orders I'm not sure and very much doubt, but Patrick's point follows in that one step at a time should be taken.


"Any fool can know. The point is to understand." -- Albert Einstein

Share this post


Link to post
Share on other sites

It would seem beneficial to save cart contents to the DB while the shopper is browsing or at least once they get into the checkout process.

 

Am I understanding correctly that osC does not do this in general? And the "PayPal IPN" contrib *does* do this? And that a future osC milestone may do this?

 

I guess there's been a lot of thread-editing recently. That could be why I'm so confused!


Patrick Breitenbach

Share this post


Link to post
Share on other sites

The cart contents for all customers are stored in a DB table. This is stock OSC functionality.

 

Orders, however are usually recorded in the DB after the customer completes payment. With the IPN contribution that I use, and that you linked to, the checkout is altered, and the order is written to the DB with a special order status *before* the customer is directed to PayPal. Then the IPN process updates this order to a confirmed order status.


-------------------------------------------------------------------------------------------------------------------------

NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit.

If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.

Share this post


Link to post
Share on other sites

Original Method

- Normally the customer would click the osC confirm button and be POSTed to PayPal.

- The customer would complete the payment and click the continue button and be returned to checkout_proccess.php where the order would then be created.

 

"PayPal IPN"

- When customer clicks the osC confirmation button they are posted to a local file called checkout_paypalipn.php

- Checkout_paypalipn.php then creates the order with a special status (PayPal Proccessing) and the storeowner has the choice whether at this time to subtract the stock from inventory.

- The customer is then redirected to PayPal via the GET method.

- Once the IPN has been received etc that order's (due to it's order_id) status is then updated - I think to one of the usual osC order states.

 

So if the customer gets to PayPal and then decides not to continue, i.e. not to pay, the order as already been generated etc....

 

The shopping cart contents are stored in 2 associated tables (customers_shopping_basket, customers_shopping_basket_attributes ) during the customers visit and checkout proceedure, and when the customer returns from PayPal (for example) it is assumed that payment has been made and deletes this information from the customers_shopping_basket db tables (this happens when $cart->reset(true) is called - after which the following session vars are removed: sendto, billto, shipping, payment, comments).

 

The holding table method would allow the order to be generated but not directly in the main orders table, this for example would enable editing of the order before it is finalized and at some point be placed into the main orders table.


"Any fool can know. The point is to understand." -- Albert Einstein

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×