429 questions
0
votes
0
answers
18
views
How to add analytics event as goal metric for ABtest?
I have created A/B tests to show two different pop-up those are variant A and baseline with each 50%. Can I add common analytics event as goal metric inside A/B test to compare both of them and ...
- 1,469
0
votes
1
answer
404
views
Split Test with VWO using URL parameters
Say I want to test users that have a specific parameter in their URL :
https://example.com/pre?q1=2&c1=67
Anyone with c1=67 passed in should then be redirected to a 50/50 split test. How can this ...
- 3,903
0
votes
1
answer
103
views
Facebook Graph API: Unable to find the created A/B Test Campaign in console UI Facebook
I am trying to create A/B test using facebook graph API. The documentation I follow: https://developers.facebook.com/docs/marketing-api/guides/split-testing
The documentation mentions following curl ...
- 135
-1
votes
1
answer
300
views
Is it possible to run Puppeteer inside another web page? [duplicate]
I am looking to create a web app which will launch a web page via puppeteer in an iframe and allow me to interact with it directly through buttons in the main page UI (not the iframe page).
Let me try ...
- 27
0
votes
1
answer
71
views
How to compare a legacy javascript method implementation with a new one?
I would like to have some easy way to support me while rewriting legacy functions/methods.
Say we have a legacy function and a rewritten one:
var legacyFunction = function() { return 1; };
var ...
-1
votes
3
answers
36
views
what is hypothesis testing?
p-value? parameter tests?
I tried to solve it but I couldn’t.
An online retail company performed an A/B test to check whether changing the size of the cart icon increases the expected number of items ...
- 1
4
votes
2
answers
210
views
Xcode 14.2 Archiving the app fails with CoreData: error: API Misuse: Attempt to serialize store access on non-owning coordinator
Archiving the app in Xcode 14.2 gives me the following error when the code does not use CoreData:
API Misuse: Attempt to serialize store access on non-owning coordinator (PSC = 0x600001a81b90, store ...
- 6,188
0
votes
1
answer
496
views
Different Algorithms for 50-50 A/B Testing
We are running A/B tests on web app customers, given a customerId. Each customer will see different minor UX changes.
Trying to prevent usage of Feature Flags as its not currently setup yet in our ...
- 1,327
2
votes
1
answer
710
views
UI not rendering when calling Statsig.updateUser
I am using the Statsig React sdk to check a gate an show some UI if the gate returns true. When I initialize with a user id that passes the gate, everything works as I would expect and the UI is ...
- 23
1
vote
2
answers
167
views
Does Google Optimize work inside an iframe hosted on an external domain?
I'm running and A/B test with Google Optimize on one of my pages.
This page is shown on multiple website on different domains through an iframe.
So, domain of my page is different from domain of the ...
- 11
-1
votes
1
answer
72
views
Google Optimize A/B Test Issue
I am currently building an A/B Test with Google Optimize that has 2 variants. The first variant displays perfectly fine on mobile/desktop and Chrome/Safari/Firefox. Variant 2 is a different story. ...
-2
votes
1
answer
74
views
Deleting duplicates in groups A/B [duplicate]
I'm conducting an A/B test and looking for an effective way to delete duplicate users ID's (visitorId column) that appear in both groups: the experiment and the control.
Here is an example:
visitorId
...
4
votes
1
answer
1k
views
Google optimize - Conversion events not being reported on server side experiment
I have setup a Google Optimize server side experiment (linked to GA4) following their documentation in this link. Once I start running the test and after a few days, I can see the experiment ...
- 191
2
votes
0
answers
403
views
Google Optimize - how experiment sessions and conversion rate are calculated?
I use Google Optimize with server-side A/B experiments. Server-side code targets audiences, consistently serves the appropriate variant to each user, etc. I only use the Optimize interface to create ...
- 10.5k
1
vote
1
answer
274
views
p-value borderline significant? Further study?
I am learning about AB testing and have run into some questions.
In the events of borderline significant p-value, say p = 0.049 and p = 0.051, is it really that different?
In the events of that I have ...
- 27
0
votes
1
answer
162
views
A/B Testing an AMP script
I need to test a amp script of an ad (tag < amp-ad >), but in AMP documentation I have only find ways to test style changes between the two versions tested.
Does anyone when through something ...
0
votes
1
answer
150
views
What exactly does A/B test mean in this scenario?
When multiple developers are continually applying changes to databases
within an organization, there may be no structured change log. Then
it's difficult to find out what change caused the database ...
- 9,886
0
votes
0
answers
146
views
A/B test adjustment only visible on new page view
During the A/B tests that I start, I often see the following happening:
I open a page that is part of the A/B test and do NOT see the variation change
I refresh the page and see the change, so I was ...
- 1
2
votes
1
answer
1k
views
ios Product Page Optimization stopped
I am using Product Page Optimization for the first time. i updated the new version of the app and the test stopped. There is no button to reactivate the test. Status header says developer has stopped, ...
- 545
2
votes
0
answers
204
views
iOS 15 Product Page Optimization info in swift
I set up Product Page Optimization on Appstore connect. I have 3 treatments and three different app icons, I want to ask can I get the current treatment name in swift so I can change app theme color ...
- 56
0
votes
1
answer
654
views
python compute sample size for one-sample Z-test
I'm trying to see if there's an easy way to calculate minimum sample size required for a one-sample Z-test to reject the null hypothesis.
I know that we can reject the null hypothesis (i.e. the A/B ...
- 15k
1
vote
1
answer
321
views
Using asyncio for doing a/b testing in Python
Let's say there's some API that's running in production already and you created another API which you kinda want to A/B test using the incoming requests that's hitting the production-api. Now I was ...
- 2,580
5
votes
1
answer
2k
views
iOS: Unable to see the option of App Icon in the Product Page Optimization page in the app store connect
As the Product Page Optimization option is now available in the App Store Connect, I thought to try it out.
So I have created a test to try different App Icons.
I have followed these steps to create a ...
- 698
2
votes
1
answer
211
views
Causal Inference where the treatment assignment is randomised
I have mostly worked with Observational data where the treatment assignment was not randomized. In the past, I have used PSM, IPTW to balance and then calculate ATE.
My problem is:
Now I am working on ...
- 676
2
votes
0
answers
44
views
Using semver to track changes to an algorithm in ruby [closed]
I'm working for a retail company that is implementing an algorithm that determines which one of our warehouses to ship orders from. The logic is going do develop over time and we need to attach the ...
- 33