Skip to content
Back to Blog

Foxy.io’s Tips for Better Security Vulnerability Reports

March 26, 2019 - Best Practices・Culture・Security

Welcome to this post! If you’ve been linked here by a Foxy team member in response to a vulnerability report, please skip the introduction. Thanks!

An Introduction

We take security seriously. Very seriously. We’re paranoid in what we believe is a healthy way, and though the PCI DSS is a really solid set of requirements, we try to go above and beyond wherever possible. (And as a PCI Level 1 Service Provider on Visa’s and MasterCard’s global registries, we’re audited annually to prove we’re taking our responsibilities seriously.)

One thing that’s helped our overall security over the years is our involvement in crowdsourced bug bounty programs on BugCrowd and HackerOne (our program there isn’t currenty public, but we’ll try to update this with a link when it goes live). As odd as it may sound, it’s wonderful to pay out for a good bug report. We consider it some of the best money we could possible spend.

That said, because we take our security so seriously, we prioritize reports and spend considerable time looking into them. That’s wonderful when the reports are well-written and clear. But unfortunately that’s not always the case. This blog post will lay out our feedback and requirements for a successful vulnerability report. If you follow these guidelines, we’ll spend less time and you’ll be more likely to earn more money.

BugCrowd and HackerOne also have multiple good resources you should review first. Our tips below aren’t exhaustive, and are intended to supplement other available resources:

Our Tips

We sincerely appreciate the time that security researchers and bug bounty hunters spend poking at our systems. We’re HAPPY to pay you for finding and reporting issues in our systems.

The following tips are our attempt to help you get paid, by us and by others, while also saving time on our end.

1. Please don’t copy/paste a generic explanation of a generic vulnerability.

TL;DR: XSS. SQL Injection. SSRF, CSRF, clickjacking, etc. We already know what those are, and we do not need a copy/pasted explanation.

We’re running a vulnerability program / bug bounty. We’re an ecommerce company. We don’t need you to explain what cross site scripting is. We need a clear explanation of what you’ve found, how to reproduce it, and how you think it’d be exploited.

Also note that if you provide only a copy/paste explanation of what an issue is, with no steps to reproduce, we will mark the report in a way to apply negative points. We don’t like to do this, but we also don’t like to waste our time.

2. Understand what you’re actually reporting.

TL;DR: If you don’t know what an issue (say CSRF or SQL Injection) is, you should not report it.

We get a surprising number of reports saying “This form doesn’t have CSRF protection! Here’s my proof of concept.” The proof of concept is an HTML form with the CSRF prevention token included.

It’s ok. We all start somewhere. But if you don’t have a solid understanding of what it is you’re reporting, spend your time learning your craft rather than burning our time.

3. Provide clear steps to reproduce the issue.

TL;DR: Provide detailed steps to reproduce an issue. Always include URLs, exact requests/responses, and tools used. A video is helpful but is not a substitute for written details.

This is not helpful:

Login form, do a GET /foo'"bar. You will see a SQL injection like this: <!doctype html><html><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content=”ie=edge”><title>…

What domain are we even talking about here? (We’ve got multiple logins across multiple systems and domains.) How are we doing a GET? In the browser? In Burp? With curl? Are we letting the browser encode the request or nah? What in the response makes you think there’s a problem? The above would be better like this:

  1. Visit https://admin.foxycart.com/ in your browser.

  2. Submit a valid username + password.

  3. Intercept the request with Burp and send to the repeater.

  4. Modify the password GET parameter value to be /foo'"bar

  5. [etc.]

Screenshots can help, as can video, but neither of those should replace a well-written text explanation. Communication is hard, but we can’t confirm what you’ve found if we don’t know how to reproduce it.

We’ll repeat this: Don’t just submit a video. We want to see text describing what you’ve done.

4. DO NOT report a result from a scanner if you haven’t verified it.

TL;DR: Just because your scanning tool says there may be an issue doesn’t make it true.

We’d prefer you not use scanning tools on our systems at all (and say as much in our program descriptions), as we already scan our systems ourselves. We know people ignore this, and if you do find something with a scanner, cool.

But please do not just send us a PDF of what your scanner says. This relates to #2 (“Understand what you’re actually reporting”). Scanning tools are often pretty stupid, and what happens is we end up wasting our time explaining to you why you didn’t actually find anything. For instance, did your scanning tool report SQL Injection or Blind Text Injection Differential? Ok, now you need to confirm that it’s actually an issue. If Burp says:

You should manually review the reported requests and responses to confirm whether a vulnerability is actually present.

The “you” there means you, not us. And if the only difference in the responses are the ETag or Last-Modified headers, you need to understand what those headers mean so you don’t report something that’s obviously not what your scanner told you it is.

5. Don’t submit multiple reports for the same issue.

TL;DR: If you find an issue that occurs in multiple places, submit only one report.

Also, do not re-submit a new report to provide additional info on an existing report. It’s just more work for us to close extra reports.

6. Don’t tell us how bad a vulnerability is.

TL;DR: We’ll know if something’s a big deal. You won’t earn more points or money if you mark a low-priority / low-impact issue as high-priority.

There are certain issues that are REALLY BIG DEALS. There are other issues that simply aren’t. If you think we’re misunderstanding the impact, please let us know. But don’t lead with “This is a critical security issue!” if you’re reporting something BugCrowd considers a P4 or P5. It sounds like you’re either being deceitful, or you don’t really understand what you’re reporting. Either way, it’s not a good look 🙂

If your heading says RCE or SQLi, that’s a “Foxy drops everything to research your submission” situation. If we drop everything only to discover a poorly written request, we’re not going to be happy about it.

Refer to BugCrowd’s taxonomy explanation if in doubt.

In summary…

We love bug bounty programs. We get a lot of really good submissions, but the majority are unclear or invalid. We can’t speak for every company, but we believe the tips above to be pretty good advice when submitting to anybody.

We hope these tips help you communicate more clearly and earn more points and money!