Use Twilio to Programmatically Text Yourself!

Twilio SMS Text Alerts
Screenshot from my Android phone of the SMS Text Messages I sent myself from Twilio (full URLs are redacted)

built a little program that checks book prices and alerts me if the price is low enough.  Recently, I integrated Twilio with it, so that I receive an SMS alert when my program finds a match.  This was surprisingly easy to do.

I’m a PHP wizard, so I wrote it in PHP, and here’s the code to do it.  That’s a github gist, and I had to alter it a bit as not to reveal any top secret code, so you’ll have to finish it a bit to meet your goals, but it should give you a good idea.  It’s fun to implement, because all you have to do is something like this:

$twilio_gateway = new Twilio_Gateway();
$twilio_gateway->text_me( 'This is an alert!' );

I love when code reads like that. So easy to understand what’s going on!

If you’ve never heard of Twilio, you should check them out.  They have APIs for incorporating text messaging, video and voice into your app.  Their developer docs are super easy to follow and they offer a free trial account!  I’m still unsure of how many text messages you get to use with a free trial account, but it’s way more than a handful.

If you’re on their trial account, text messages will include a notice that they were sent from a trial account.  You also can only send messages to phone numbers that you have verified.  That said, if you want to send alerts to yourself for free, Twilio makes that possible.  And if you decide to upgrade to a paid account, it seems that the cost is still going to be pretty cheap for personal use.  I first heard of Twilio while listening to a Software Engineering Daily podcast, and the ad said it was something like a penny a message.  You do have to buy a phone number to send the messages from, but that’s also super cheap.

SMS is a great option for sending alerts, because modern smartphones allow you to designate different notification sounds for each number, so I have a unique sound for when it’s an alert from my program.  The trick is to make sure that the alert only comes in when it’s actually important, as when alerts happen too often, the human brain tends to tune them out.  So the next step is optimizing my program to get better at figuring out when something really is a deal!

Debugging PhpStorm with xDebug on Ubuntu – Solving Common Set-up Problems

Do you love programming in PHP?  If you don’t, could it be your editor/IDE?

As a programmer, the tools you use, and how well you use them, can have a huge impact on your productivity.

How much time have you wasted writing var_dump statements and printing log messages?

What if you could just inspect the contents of a variable whenever you wanted?

That’s what debugging allows you to do, and that’s what I do every day.

Not sure what the array keys are?  Inspect the variable.  Not sure of the object’s properties?  Inspect the variable.  Not sure if a function is running?  Stick a breakpoint on its first line, and see if your program stops execution at that point.

Debugging saves time… But setting up debugging can also take some time, as it requires the installation of xdebug and the correct configuration set-up.  Plus, if you’re using a virtual machine like a vagrant box, you’ll need to set up path mappings.

PhpStorm Has Built-in Debugging for xdebug

See that screenshot?  If you’re trying to get xdebug to work with phpstorm, and you’ve already installed xdebug, then you should go to: Run –> Web Server Debug Validation (For PhpStorm version 2016.3.2, it’s the last option in that Run dropdown menu.)

When that validation box first opens, you’ll probably see an error message.  Click on “validate” and it might work right out of the box. If not, check that the path is set to a location that the web server can serve up. If you already have a project, this can just be the directory to the project.  (By default, Apache only serves up files from within a specific directory, and as programmers we usually put our web apps inside that directory, because then we can view them with a web browser.)

Settings for Ubuntu 16.04 with Apache and PHP 7

If you’re running Ubuntu with Apache and PHP locally, make sure that “Local Web Server or Shared Folder” is selected, and enter the following:

Path to create validation script: /var/www/html/

URL to validation script: http://127.0.0.1/

Click Validate….. and PhpStorm will tell you what’s wrong!!

In this case, it was easy.  I just had to add xdebug.remote_enable=1 to my php.ini file

And then I just had to restart Apache:

systemctl restart apache2

Hope that was helpful!

 

Which PHP Framework should I learn? (2016 Comparison)

I’ve become interested in learning a PHP Framework (in addition to the WordPress CMS) for two reasons.  First, I’ve been looking at PHP job listings and experience working with at least one framework is required for most mid to senior level developer/software engineer jobs.

The second reason I’ve decided to learn a PHP framework is that I have an idea for expanding my Amazon Search program and I like the advice to “always be learning something new.”  I think this advice comes from the book Coders at Work by Peter Seibel, but it’s possible that the advice is really from some other place… But it makes sense that I might as well improve my skills while I’m also improving my application.

So I’ve been spending my last few evenings drawing UML class diagrams (as well as flow charts) to plan out my program’s design.  I also spent a day getting the Laravel framework set up on my development server.  So far, I’ve set up a Laravel program that does basic CRUD and I can see why people want to work with Laravel.  The hardest part, after initial set-up, was not overthinking things!  Laravel has a lot of “syntatic sugar” so the code is easy to write and read, but debugging is a pain because functions are hidden inside of traits and behind facades.

Before choosing Laravel for my project, I did some research on PHP Framework popularity.  I searched the job hunting website Indeed.com for major frameworks along with the keyword PHP.  This is what I found:

Framework Popularity based in Indeed Job Openings

For that graph, I omitted the frameworks with 20 or less mentions, which were Aura, Apigility, and Phalcon.

As you can see, I included both Model-View-Controller (MVC) frameworks as well as Content Management Systems (CMS).   While some applications could use either type of framework, in general a CMS is designed for a blog type site or a site managing a lot of content.

For CMS Frameworks, WordPress beat Drupal by a few hundred listings.  This is for a nationwide search that also included the keyword PHP. This was to hopefully exclude jobs that weren’t true PHP programming jobs (like WordPress front-end design or WordPress website management or WordPress Marketing or WordPress SEO), but because this search included jobs that list PHP as a “nice to have” skill, the actual number of WordPress backend PHP programming jobs is probably significantly lower.

As for the MVC frameworks, Laravel and Zend were the most in-demand, each having over 700 job openings.  One word of caution about this result is that some job lisitngs state something like this “Requires knowledge of an MVC PHP Framework, such as Laravel or Zend.”  In this case, they are just giving an example of a popular one, but if you were familiar with CakePHP or CodeIgniter (both MVC Frameworks), then you would also meet that job qualification.

What PHP framework should I use?

Ultimately, you should either search for local jobs and find companies that you want to work for, or search for open source projects that you are interested in, and use the same frameworks that they are using.

Tip: You can use the website builtwith.com to find out what frameworks and other technology a website is using.

An experienced programmer can learn a new framework with relative ease, so once you have strong experience with one framework, you can probably quickly learn another if needed.

What PHP framework should I LEARN?

Another option for choosing a framework is to look at the availability of tutorials and books.  If you can find a tutorial or book that covers the topics you’ll need for your project, in a learning style that you like, then perhaps you should use that framework.  Finding a good instructor or mentor is GOLD!   So, find the best source for learning whatever it is that you want to do, and then use the same frameworks and tools that they are using.  There are a plethora of code instruction websites that have video tutorials and online courses.  Most do cost something, but I’ve found free access to Lynda.com through my local library, Udacity has deeply discounted sales regularly, and Pluralsight gave me a couple of months free as a promotion, so if you join the mailing lists of these educational sights, you might find a low-cost deal.

 

Do I NEED a PHP Framwork?

Like most everything, there’s both positives and negatives.

Positives:

  •  A good framework will save you time because it provides basic functionality common to
    many programs so you don’t have to reinvent the wheel!
  •  MVC Frameworks help with code organization.
  •  A framework may make it easier for other programmers to understand your code.
  •  A well-established, open source framework means more eyes on the code and lots of
    bugs already fixed

Negatives:

  • Frameworks can take awhile to set-up and learn.
  •  It may be harder to debug code if you don’t understand the framework thoroughly.
  • Applications may be slower and larger if the framework includes a lot of code that is not relevant to your project.
  •  Your application will depend upon the framework’s continued support and updates (you’ll have to update the framework, especially if security issues are discovered.)

 

How to End an Ebay Listing Automatically with a PHP Script

One programming project I completed included a custom backend order management system.  The client had a website where she sold items that were also listed on Ebay.   She had been manually removing the items when they sold by logging into her Ebay account and going to the Ebay End Your Listing Early Page (this can be found on the Ebay Site Map).  Of course, doing this manually has a few issues.  Not only is it time consuming, but it also runs the risk of an out-of-stock item selling.  For best ecommerce practices, store inventory should be updated immediately.  So automating this task can really save a lot of headaches!

This backend sales automation task is fairly simple.  It involves sending an XML document to Ebay and then parsing the XML response that they return.  It transfers the data to and from Ebay’s server using the php cURL library, which is very commonplace.  You can check to see if your server has cURL enabled by using

function_exists('curl_version')

If you’re working with a WordPress plugin, I suggest that you use the WordPress HTTP API instead, as it has alternative ways of completing the HTTP request if cURL doesn’t exist.

If you’re writing a program to simply handle one Ebay seller’s inventory, you can get the necessary authentication token by signing up for the Ebay Developer Program.  The only other piece of data you need is the Ebay Id, which in my case was stored in a custom MySQL Database table.

This is the PHP script that will remove the Ebay listing:



/**
 * This function ends an Ebay listing.  Returns true if successful.
 *
 * @param $ebay_id String Ebay Item Id
 * @param $auth_token String EbayAuthToken
 */
function remove_item_from_ebay( $ebay_id, $auth_token ) {
	$endpoint = "https://api.ebay.com/ws/api.dll";

	$xmlbody = '
<?xml version="1.0" encoding="utf-8"?>
<EndItemsRequest xmlns="urn:ebay:apis:eBLBaseComponents">
	<RequesterCredentials>
		<eBayAuthToken>' . $auth_token . '</eBayAuthToken>
	</RequesterCredentials>

	<!-- Call-specific Input Fields -->
	<EndItemRequestContainer>
		<EndingReason>NotAvailable</EndingReason>
		<ItemID>' . $ebay_id . '</ItemID>
		<MessageID>' . $ebay_id . '</MessageID>
	</EndItemRequestContainer>
	<!-- ... more EndItemRequestContainer nodes allowed here ... -->
	<!-- Standard Input Fields -->
	<ErrorLanguage>en_US</ErrorLanguage>
	<WarningLevel>High</WarningLevel>
</EndItemsRequest>
';
	
	$headers = array(
//Regulates versioning of the XML interface for the API
		'X-EBAY-API-COMPATIBILITY-LEVEL: 861',
//the name of the call we are requesting
		'X-EBAY-API-CALL-NAME: EndItems',
//SiteID must also be set in the Request's XML
//SiteID = 0  (US) - UK = 3, Canada = 2, Australia = 15, ....
//SiteID Indicates the eBay site to associate the call with
		'X-EBAY-API-SITEID: 0 ',
	);


//initialise a CURL session
	$connection = curl_init();
//set the server we are using (could be Sandbox or Production server)
	curl_setopt( $connection, CURLOPT_URL, $endpoint );

//stop CURL from verifying the peer's certificate
	curl_setopt( $connection, CURLOPT_SSL_VERIFYPEER, 0 );
	curl_setopt( $connection, CURLOPT_SSL_VERIFYHOST, 0 );

//set the headers using the array of headers
	curl_setopt( $connection, CURLOPT_HTTPHEADER, $headers );

//set method as POST
	curl_setopt( $connection, CURLOPT_POST, 1 );

//set the XML body of the request
	curl_setopt( $connection, CURLOPT_POSTFIELDS, $xmlbody );

//set it to return the transfer as a string from curl_exec
	curl_setopt( $connection, CURLOPT_RETURNTRANSFER, 1 );

	curl_setopt( $connection, CURLOPT_TIMEOUT, 5 );
//Send the Request
	$response = curl_exec( $connection );

//close the connection
	curl_close( $connection );

	$r = simplexml_load_string( $response );

	return $r->Ack == 'Success';
}

You can also grab this code from my Github account.

I may improve this code in the future so that it’s more informative.  I could have it throw an exception on failure, or I might create a class that allows the error to be retrieved.  The response from Ebay also contains the ebay ID (in CorrelationID XML Tag), so for further validation you could verify that the ID is what you expected.

When parsing this response, remember that the root element “becomes” the SimpleXML Object.  In this case, the root element is EndItemsRequest.

A common error is to try to refer to this root element:

$r->EndItemsRequest->Ack   // WRONG
$r->Ack // CORRECT

For quick reference, here’s what the response XML looks like:

    <?xml version="1.0" encoding="UTF-8"?>
<EndItemsResponse xmlns="urn:ebay:apis:eBLBaseComponents">
    <Timestamp>2015-12-16T15:36:29.791Z</Timestamp>
    <Ack>Success</Ack>
    <Version>949</Version>
    <Build>E949_UNI_API5_17774433_R1</Build>
    <EndItemResponseContainer>
        <EndTime>2015-12-16T15:36:29.000Z</EndTime>
        <CorrelationID>222253987187</CorrelationID>
    </EndItemResponseContainer>
</EndItemsResponse>

 

And, also for easy reference, here is an example of what an Ebay failure response looks like:

<EndItemsResponse xmlns="urn:ebay:apis:eBLBaseComponents">
    <Timestamp>2014-09-25T18:52:41.125Z</Timestamp>
    <Ack>Failure</Ack>
    <Errors>
        <ShortMessage>Errors in Input Data.</ShortMessage>
        <LongMessage>Errors in Input Data.Please try again.</LongMessage>
        <ErrorCode>400</ErrorCode>
        <SeverityCode>Error</SeverityCode>
        <ErrorClassification>RequestError</ErrorClassification>
    </Errors>
    <Version>891</Version>
    <Build>E891_UNI_API5_17049963_R1</Build>
    <EndItemResponseContainer>
        <CorrelationID>290345600169</CorrelationID>
        <Errors>
            <ShortMessage>The auction has been closed.</ShortMessage>
            <LongMessage>The auction has already been closed.</LongMessage>
            <ErrorCode>1047</ErrorCode>
            <SeverityCode>Error</SeverityCode>
            <ErrorClassification>RequestError</ErrorClassification>
        </Errors>
    </EndItemResponseContainer>
</EndItemsResponse>

 

Hopefully, this little bit of code will help someone else struggling with how to implement this task.   Since my client was getting payment by Paypal, I used it in a IPN script (Instant Payment Notification).  I also had my IPN script send email notifications with the order information.  Paypal only automatically sends payment notifications to one email address, so if you want to have email notifications sent to multiple address, implementing Paypal’s IPN is one solution.

Improving on This: From PHP Script to PHP Class

Although procedural-style code is common for the PHP programming language, object-oriented programming for the web with PHP is becoming more popular.  Regardless of whether you write OOP style code, creating classes is a good idea for organization.  You can put similar functions together.  Also, using classes really decreases the chance that you’ll run into a naming conflict.  You only have to make sure your function name is unique to the class and not to the whole codebase.

From an object oriented perspective, you want to name your classes after nouns.  So I named my class EbayConnection, although EbayConnector would also work, but naming your class GetEbayConnection would be incorrect.  getEbayConnection (with a lowercase g, by standard conventions) would be a function name, or, when in a class, a method name.   Methods and functions are pretty much the same thing.  It’s just that we only call them methods when they are in a class.  And Methods should usually contain verbs.  At least that’s that I learned — it makes reading code easier and it helps in understanding how the program works!

So, I wrote the class EbayConnection (Grab it on Github).  I expanded on the above function not only by refactoring it a bit, but I also added some error reporting so that the calling program can now get the last error message and last error code.

class EbayConnection {
	private $auth = null;
	private $error_msg = null;
	private $error_code = null;
	function __construct($auth) {
		$this->auth = $auth;
	}
	/**
	 * This function ends an Ebay listing.  Returns true if successful.
	 *
	 * @param $ebay_id String Ebay Item Id
	 */
	public function endItem($ebay_id) {
		$endpoint = "https://api.ebay.com/ws/api.dll";
		$xmlbody = '
		<?xml version="1.0" encoding="utf-8"?>
		<EndItemsRequest xmlns="urn:ebay:apis:eBLBaseComponents">
		<RequesterCredentials>
		<eBayAuthToken>' . $this->auth . '</eBayAuthToken>
		</RequesterCredentials>
		<!-- Call-specific Input Fields -->
		<EndItemRequestContainer>
		<EndingReason>NotAvailable</EndingReason>
		<ItemID>' . $ebay_id . '</ItemID>
		<MessageID>' . $ebay_id . '</MessageID>
		</EndItemRequestContainer>
		<!-- ... more EndItemRequestContainer nodes allowed here ... -->
		<!-- Standard Input Fields -->
		<ErrorLanguage>en_US</ErrorLanguage>
		<WarningLevel>High</WarningLevel>
		</EndItemsRequest>';
		$headers = array(
			//Regulates versioning of the XML interface for the API
			'X-EBAY-API-COMPATIBILITY-LEVEL: 861',
			//the name of the call we are requesting
			'X-EBAY-API-CALL-NAME: EndItems',
			//SiteID must also be set in the Request's XML
			//SiteID = 0  (US) - UK = 3, Canada = 2, Australia = 15, ....
			//SiteID Indicates the eBay site to associate the call with
			'X-EBAY-API-SITEID: 0 ',
			);
		//initialise a CURL session
		$connection = curl_init();
		//set the server we are using (could be Sandbox or Production server)
		curl_setopt( $connection, CURLOPT_URL, $endpoint );
		//stop CURL from verifying the peer's certificate
		curl_setopt( $connection, CURLOPT_SSL_VERIFYPEER, 0 );
		curl_setopt( $connection, CURLOPT_SSL_VERIFYHOST, 0 );
		//set the headers using the array of headers
		curl_setopt( $connection, CURLOPT_HTTPHEADER, $headers );
		//set method as POST
		curl_setopt( $connection, CURLOPT_POST, 1 );
		//set the XML body of the request
		curl_setopt( $connection, CURLOPT_POSTFIELDS, $xmlbody );
		//set it to return the transfer as a string from curl_exec
		curl_setopt( $connection, CURLOPT_RETURNTRANSFER, 1 );
		curl_setopt( $connection, CURLOPT_TIMEOUT, 5 );
		//Send the Request
		$response = curl_exec( $connection );
		//close the connection
		curl_close( $connection );
		$r = simplexml_load_string( $response );
		if($r->Ack == 'Success') {
			return true;
		} else {
			$this->recordError($r);
			return false;
		}
	}
	private function recordError($response) {
		if(isset($response->Errors->LongMessage)) {
			$this->error_msg = $response->Errors->LongMessage;
		} elseif(isset($response->Error->ShortMessage)) {
			$this->error_msg = $response->Errors->ShortMessage;
		}
		if(isset($response->Errors->ErrorCode)) {
			$this->error_code = $response->Errors->ErrorCode;
		}
	}
	/**
	 * @return null|String
	 */
	public function getErrorMsg() {
		return $this->error_msg;
	}
	/**
	 * @return null|String
	 */
	public function getErrorCode() {
		return $this->error_code;
	}
}

Sanitizing & Validating User Input for Amazon Product Advertising API

Summary:  Remove apostrophes to prevent Amazon from returning “No Results”

As a security-conscious web programmer, I always sanitize user input. It’s something you have to do to prevent attacks like SQL Injection and Cross-Site Scripting.  Basically it means that you never trust user input to be safe, and you always filter or sanitize it to remove potentially dangerous input.   PHP has lots of filters for sanitization, so that part is easy.  For Amazon item search, my PHP code filters the user input like this:


if(isset($_POST['title']) && !empty($_POST['title'])) {
     $search_values['Title']= filter_var($_POST['title'], FILTER_SANITIZE_STRING);
}

But there’s a second step.   In addition to sanitizing data, you also want to validate it to make sure that it’s in the expected format.  But, in order to validate correctly, you have to understand the rules about what’s expected. Sometimes the rules are obvious, but other times you’re working with a bit of a black box.

Third party APIs are like black boxes.  You can’t read the code.   You only have the documentation to rely upon.   Giving the wrong input will get you errors or no results, and you might not know what you’re doing wrong.  So, you need to experiment and see if you can figure out what’s allowed and what’s not.

This post is specific to the Amazon Product Advertising API, which is an Amazon service that allows you to look up product information, like availability, pricing, keyword search, etc.

I use the Amazon Product Advertising API for several different projects, and for one of them, my application uses the ItemSearch operation. Specifically, I often search the Books category by Title.

Amazon’s API behaves differently than the search function on their site.  On their site, if you put in an apostrophe, the site just ignores it.  When using the API, you get no results.

So, the solution is to always strip the user input of all apostrophes. Or, if you’re a fellow programmer, you might call them single quotes. If you leave the apostrophe in, you’ll get no results. If you remove it, you hopefully get lots of results!

In my program, I am sending the user input via an HTML Form using Ajax.  My browser actually encodes the user input before sending it to the server.  It encodes the apostrophe character as its html entity which is &#39;.

So, in my back end PHP code, I remove the apostrophe with this:


str_replace("&#39;", "", $my_search_string);

Problem solved!   In actuality, I have combined that function with my code for sanitization, and the whole thing looks like:

 


if(isset($_POST['title']) && !empty($_POST['title'])) {
     $search_values['Title'] = str_replace("'", "", filter_var($_POST['title'], FILTER_SANITIZE_STRING));
}