Apparently, science takes quite a few salvages (6k and counting...).
After opening 1000 bags to check whether they're affected by diminishing returns / magic find (spoiler: they aren't), I started wondering about salvaging. In particular, Lord Signis found evidence that "items may have their own rare salvage modifier value other than 1. (ie .52 from the item and .25 from the salvage kit means salvaging will yield rare materials 13% of the time) Further testing is needed."
Challenge accepted.
I bought 1500 rags from the trading post. Because I apparently have no desire to wear exotics. Ever.
To salvage the rags, I purchased 500 Crude, Basic, and Fine salvage kits. I recorded the number of silk and gossamer from each salvage, and appended this to Signis's data on 500 salvages with Mystic salvage kits.
For reference: Crude kits have no listed chance of giving rare items, a 10% chance for Basic kits, 15% for Fine kits, and 25% for Mystic kits.
Data can be found here.
As before, I see no effect from the anti-bot code. Salvage away without diminishing returns.
Visually, here is the average number of silk and gossamer from each salvage, by number of items salvaged:
The average gossamer yield from each kit was:
Crude: 8.6% (1.3%)
Basic: 13.0% (1.5%)
Fine: 12.8% (1.5%)
Mystic: 13.0% (1.5%)
(Standard errors in parenthesis.)
So, while crude kits had an 8.6% chance of gossamer, all other kits had a 13% chance.
While it is always possible that I was exceptionally unlucky with crude kits... it is exceptionally unlikely. The probability that I'd observe gossamer 8.6% of the time, when the true chance was 13%... is about 2%. So possible. But not likely.
This contrasts with the claims in this video of "100% certainty" that crude and mystic have identical salvage rates. Hopefully he'll be willing to share the data, and I can take a look at what is going on.
The crude kits leave me a little baffled. Without them, I'd say that gossamer had a 13% chance of being salvaged from a rag and that kits don't matter. With them... kits matter (at least on the choice of crude vs non-crude). But the rate doesn't match what is listed on the item. Hopefully I can salvage these results over the weekend. I've put in more orders for rags to check whether my crude data is just an unlucky draw, and started to collect other salvageable items to check those too.
TLDR: Salvaging rags with Basic, Fine and Mystic kits has a 13% chance of giving gossamer. Crude kits have an 8% chance. More testing required.
Update: I've done another 500 salvages with crude kits... still at about 8.7%. Now going to check other materials. I've decided to make it clearer that the I'm presenting standard errors of the means here, because otherwise I find +/- ambiguous / unfounded.
crude kits - 24 gossamer so its 9,6%
ReplyDeletebasic kits - 13 gossamer - 5,2%
journe kits - 21 gossamer - 8,44%
mystic kits - 22 gossamer - 8,88%
salvaged 250 rags for each test
Sometimes, I think we need a guild of data nerds in GW2 to share stuff like this...
DeleteI'm looking at "Average Gossamer per salvage diagram".
ReplyDeleteWhat I'm seeing is despite the "small" sample size, a clear convergence towards a constant same value for all three kits except the crude can be observed. Or is the sample size not large enough do draw this conclusion? Doesn't it seem rather VERY unlikely that these three curves would actually split up and break the trend each converging to their own individual set even given a sample size of >> 500? Maybe there is something I'm not seeing :) . But what if so?
If the three kits are in fact converging to the same value then this means either of two things:
1. The "standing" theory in the community that kits give have a prob of yielding higher tier mats that is
ProbForHighTierYield = BasicProb * ( KitRarerProb + 1 )
must be wrong.
2. The mechanic here is bugged.
Those are my conclusions at time of writing based on what I'm seeing.
As I stated in another place I'm very impressed with and intrested in what you're doing here. I hope you take this input "the right way", I'm basically just having fun discussing alternatives and want to aid in nailing down what's going on here :)
You could totally draw that conclusion. It'd be reasonable to say that all tested kits except crude share approximately the same salvage rates of gossamer, and that crude is a low outlier.
DeleteWhat troubles me with that conclusion is that others have reported that after thousands of salvages of rags with crude kits, they got gossamer 11% of the time. (I've asked for the data =D!) Assuming thats true, you'd expect to see results as extreme as mine or worse 2% of the time. So I'd be unlucky, but not crazy unlucky.
The bigger trouble is, with the amount of data I have and the effect we're looking for, I just don't have enough power with the tests to distinguish between the Multiplicative Chance Theory vs Totally Bugged Mechanic.
Thanks for pushing against the results by the way. I've become a bit unsettled that my overenthusiastic statement that "the data are consistent with that" got translated into "definitely works that way". Its going to take a ton more testing to really sort this out.
The only reason I would be wary about using these results to try to draw a conclusion about the listed percentages on the salvage kits is that Gossamer, while "rarer" than Silk for player purposes, is still a Common-quality material. For all we know, the percentages listed on the salvage kits actually refer only to Rare-quality crafting materials and above (Lodestones, Ectoplasm, Skulls, Doubloons, etc).
ReplyDeleteSimilarly, diminishing returns may not apply to Common-quality materials; the base rate of Gossamer to Silk may be simply based on the item level of the salvage material, though the difference between the crude salvage kit and the others is interesting and noteworthy.
I think you have some interesting ideas about salvaging to obtain rare-quality crafting items. But that isn't what I'm examining here. Right now, I'm focusing on understanding a bit more about salvaging common-quality items to obtain common-quality crafting materials. Once that is sorted (which takes a fair bit more salvaging), I'll try to build on those results to study other things.
DeleteSide note: Have you ever obtained a rare+ quality crafting item - other than an ecto - from a salvage? If so, do you remember the item that was salvaged?
This comment has been removed by the author.
ReplyDeleteHello, Lord Signis here. I hope my google account is connected to a Signis name. I did this same experiment some time ago and was unable to publish my results due to issues that came up in real life. I can confirm these results.
ReplyDeleteHey! I hope RL stuff is well - I'd been trying to get in touch with you in-game about testing various mechanics out. (I think we're friended - my name is obvious?)
DeleteDid you also get that crude was lower than the others, which were essentially indistinguishable?
... and did you keep the data? =D
DeleteI will see if I can recover the raw data but the chances are very slim. Like you, I did detect small variations in the returns rate but had concluded that, functionally, (when salvaging Rags), salvage kit doesn't much matter assuming you use a basic or higher quality. I did not test Crude because I was testing for an internal value and because the percentage is not listed on the tooltip I decided to avoid that. It seems to follow a linear trend (Caveat on that later) with the equation of y(real salvage rate)=.0267x(Kit rate) + 12.333 (repeating, of course). While this almost falls in line with your observations, the crude salvage kits are an enigma. If linear (which would make the most sense) the salvage kit value for crude would be about -138... Perhaps it is a square root function. REGARDLESS, unless someone is salvaging tens of thousands of rags they will not notice a statistically significant difference between kits other than crude.
Delete