Cannot estimate paths when account has many trustlines

I'm trying to find paths from BTC to XLM
Here's the query
I'll post example result here, because it changes over time
{
"_embedded": {
"records": [
{
"source_asset_type": "native",
"source_amount": "1000.0000000",
"destination_asset_type": "native",
"destination_amount": "1000.0000000",
"path": []
},
{
"source_asset_type": "credit_alphanum4",
"source_asset_code": "ICN",
"source_asset_issuer": "GBDEVU63Y6NTHJQQZIKVTC23NWLQVP3WJ2RI2OTSJTNYOIGICST6DUXR",
"source_amount": "11.9476800",
"destination_asset_type": "native",
"destination_amount": "1000.0000000",
"path": []
},
{
"source_asset_type": "credit_alphanum4",
"source_asset_code": "LTC",
"source_asset_issuer": "GC5LOR3BK6KIOK7GKAUD5EGHQCMFOGHJTC7I3ELB66PTDFXORC2VM5LP",
"source_amount": "0.3664700",
"destination_asset_type": "native",
"destination_amount": "1000.0000000",
"path": []
},
{
"source_asset_type": "credit_alphanum4",
"source_asset_code": "BAT",
"source_asset_issuer": "GBDEVU63Y6NTHJQQZIKVTC23NWLQVP3WJ2RI2OTSJTNYOIGICST6DUXR",
"source_amount": "103.8522300",
"destination_asset_type": "native",
"destination_amount": "1000.0000000",
"path": []
},
{
"source_asset_type": "credit_alphanum4",
"source_asset_code": "REP",
"source_asset_issuer": "GBDEVU63Y6NTHJQQZIKVTC23NWLQVP3WJ2RI2OTSJTNYOIGICST6DUXR",
"source_amount": "1.0196900",
"destination_asset_type": "native",
"destination_amount": "1000.0000000",
"path": []
}
]
}
}

Horizon limits number of returned paths to 5
and since I have many trust lines there is a good chance that BTC won't be present in the result set.
Increasing or making limit a parameter is dangerous for performance, so I assume sourceAsset optional parameter is required to be able to estimate paths between assets of my choice.

look at potential design to fix this

After looking into this.. we are currently getting all the assets available to the source account as inputs to the pathfinder query (takes in an array of source assets). We can very easily limit this to a single source asset based on new optional url params. This will also be backwards compatible.

how expensive is it to raise the limit to say 20?

I imagine I'll have more than 20 tokens anyway, so either there should be a way to specify, or I'll be using some dummy accounts with few trustlines each (doesn't require anything from your side)

specifying seems weird to me since I'm not sure how the user would know what to specify? I think we need paging ultimately

User may not know, but wallet knows what assets user has, I don't see a problem to have that as an option.
Yeah, paging is an option, but I guess having paging I'd still try to avoid it in my implementation, sending 3 consequent requests introduces big delay


Automatically expanding keywords using --expandkeywords does not function

The cli argument --expandkeywords does not seem to be working for me.
Relevant documentation link
Robot version: Robot Framework 5.0.1 (Python 3.10.7 on win32)
Steps to replicate:
using file ExampleSuite.robot:
*** Test Cases ***
Test that should be expanded
    [Tags]    expand
    Log    Now you see me

Test That should not be expanded
    Log    Now you don't!

$ robot --expandkeywords TAG:expand .\ExampleSuite.robot
The log output does not have the tagged test case expanded by default.

Whoops, test != keyword.

Yeah, this behavior is by design because expanding wors based on keyword tags. If there are use cases where looking for test tags would be useful, that enhancement could be considered.


Splash Screen

AccountActivity is delayed in showing up. Users get to see the actual app for a second. Implement a splash screen to allow the use of SharedPreferences to either load AccountActivity or MainActivity

Added blank splash now


JAVA-1935: Add null check in QueryConsistencyException.getHost

For JAVA-1935

I commented on the Jira ticket: I think ti would be wiser to amend all subtypes of CoordinatorException, not only this one, wdyt?

Good catch, I didn't make that connection.  Will update all the applicable classes.


fixing path check/create for script only

Type of Change

[x] Bug fix (non-breaking change, fixes #4046 )
[ ] New feature (non-breaking change, adds functionality)
[ ] Breaking change (effects multiple commands or functionality)
[x] Ran manual Pester test and has passed (`.\tests\manual.pester.ps1)
[ ] Adding code coverage to existing functionality
[x] Pester test is included
[ ] Nunit test is included
[ ] Documentation
[ ] Build system

Purpose
OutputScriptOnly wasn't shorting out the folder tests.
Approach
Now it does skip them. An info message about the missing path is raised in the verbose stream

thanks so very much!


[SPARK-45705][TESTS] Fix flaky test: Status of a failed DDL/DML with no jobs should be FAILED

What changes were proposed in this pull request?

Fix flaky test: Status of a failed DDL/DML with no jobs should be FAILED
Why are the changes needed?

- SPARK-44334: Status of a failed DDL/DML with no jobs should be FAILED *** FAILED *** (150 milliseconds)
[info]   "[RUNNING]" did not equal "[FAILED]" (SqlResourceWithActualMetricsSuite.scala:148)
[info]   Analysis:
[info]   "[RUNNING]" -> "[FAILED]"
[info]   org.scalatest.exceptions.TestFailedException:
[info]   at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
[info]   at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
[info]   at org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1231)

There could be contention on between UI updates and test asserts
Does this PR introduce any user-facing change?

no
How was this patch tested?

fixed tests
Was this patch authored or co-authored using generative AI tooling?

no

cc @LuciferYang @HyukjinKwon @dongjoon-hyun thanks

thanks, merging to master

late LGTM


ads not blocked

here is url of the ads that not blocked by ddg:
https://click.a-ads.com/1648168/100233/
this ads appear while I visit this web:
https://techyneed.com/wp/how-safe-is-your-job-here-are-7-signs-that-are-career-red-flags/
well pls add click.a-ads.com to blacklist so next time this ads are blocked by ddg browser @duckduckgo
thanks 😊

Hey @NSTAdventure, thanks for taking the time to write this.
We block ads that we consider to be tracking our users. For that, we consume the dataset provided by https://github.com/duckduckgo/tracker-radar If you think that domain should be added to the block list, feel free to ping the team in that repo. Thanks.

Hey @NSTAdventure, thanks for taking the time to write this.
We block ads that we consider to be tracking our users. For that, we consume the dataset provided by https://github.com/duckduckgo/tracker-radar If you think that domain should be added to the block list, feel free to ping the team in that repo. Thanks.

wait don't close this yet i will try to report that just wait :)


Different behavior between { headless: false } and { headless: true }

I'm curious to know what changes there are between running as headless true vs false. When I run a login to Amazon using headless: true I get an error from Amazon via the screenshot. But when I set headless: false I watch it work just fine, no error.
So I'm trying to figure out what headless: true is doing that is different from when it's not headless.
Thanks to any suggestions.

There could be any number of things going on. They could be looking for the Headless added to the UA string and blocking that. Or they could be using some techniques to detect automated access and prevent it.
If it works in non-headless and fails in headless then the site itself is doing something to prevent automated access. So you'd need to figure out what that is and work around it or move on. Some things are easy to get around (like modifying the UA string) while others are non-trivial to bypass.

I am also facing the same issue.
When Headless is false
page url ===> http://lvh.me:3000/dashboard
When Headless is true
page url ===> about:blank
(node:29206) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1)

Can anyone provide an actual example file to run that reproduces this issue?

I will try to find something public that I can post. My example is confidential so I can't share it.

@Garbee just FYI, I'm setting the UA, so I don't think that's it. And I'm performing things like delays and mouse movement etc. Since the only difference is the headless: true it leads me to believe that there is something going on in the lib, and not on the site that I'm scraping. But I will keep trying and hopefully will find an example to post.
Are there other kinds of debugging maybe that can help point to where an issue might be?

@Garbee Here is code. This happens only for localhost if I give the actual website URL (http://www.google.com... etc) it is working for both options.
const browser = await puppeteer.launch({headless: true});
  const page = await browser.newPage();
  await page.goto('localhost:3000', {
          networkIdleTimeout: 1000,
          waitUntil: 'networkidle',
          timeout: 3000000
        });
  console.log(page.url());

Output:
about:blank
Expected output:
localhost:3000
If headless is false I am getting the expected output.

I'll thicken the plot. I've started debugging the POST requests to my amazon login. When headless is set to true, Amazon is making an additional POST request that I don't recognize. That doesn't exist when headless is set to false. So that says to me something else is changing with this setting that I don't yet know.

I've also inspected the request and response for both headless and non-headless. They seem to be identical in nature.

In non-headless mode, screenshots work differently because my screen is in HiDPI mode (MacBook Retina). Here's one of the example screenshots:

Remember the protocol is required for urls in goto.
@LoganDark that is a different issue completely. Please file your own for triage and discussion.

Different issue? Well, I didn't know that because of the title.

Reading the issue description as well, nothing stands out to me that would make my issue completely different. Here are the parts that made me think my issue did belong here:

I'm curious to know what changes there are between running as headless true vs false.

So I'm trying to figure out what headless: true is doing that is different from when it's not headless.

@Garbee Yes giving the protocol in goto solves the issue.
await page.goto('http://localhost:3000', {
          networkIdleTimeout: 1000,
          waitUntil: 'networkidle',
          timeout: 3000000
        });
console.log(page.url)

If I don't give the protocol for google.com, am getting an error Error: Protocol error (Page.navigate): Cannot navigate to invalid URL undefined whereas for the above case I am getting about:blank. The error handling it done differently for localhosts.. Shouldn't it be giving the protocol error?
await page.goto('www.google.com', {
          networkIdleTimeout: 1000,
          waitUntil: 'networkidle',
          timeout: 3000000
        });
console.log(page.url)

@LoganDark Sorry about the poorly worded title for the issue. There is nothing I can do about that. Your issue is with screenshot functionality while this was opened about some navigational problems. They are entirely distinct separated issues. Therefore a new issue is required to focus on your problem.
@kaushik-sundar Throwing an error for missing the protocol is a good idea IMO. I'll need to look into it though as it could be non-trivial to setup well due to the number of allowed protocols.

My apologies on the title, but I do agree that protocol issue is separate. My issue is more related to something about the request from the browser is different when headless is on vs off, causing the site in question to act differently.

Here is a gist of the problem.  With params.isHeadless as false the browser opens and the form successfully, whereas with it false I get an auth error page (which I actually cannot replicate through normal means no matter what kinds of correct/incorrect credential permutations I try to use).
Since the problem is behind an auth wall (or rather, the act of authenticating itself) I cannot share the exact code with my own credentials.  However if you have or create your own vendorcentral account you should be able to see this behavior.
I wrote the code in such a way that it works for some other services as well, such as imgur.  For this, just change params.url (to https://imgur.com/signin for example).  It works on Imgur, which implies that Amazon is doing something explicit, however we have been as of yet unable to determine what that is, because as @optikalefx has said we have tried sporadic mouse movement, delayed typing, etc.

Note: I'll open another issue for this eventually as I need to do more research and experimentation, but I found that page.press('Enter') does not actually press the enter key.  At least for me and my environment.

Try page.press('Return') as well..?

@LoganDark That didn't work either. I probably shouldn't have brought it up here at all, completely unrelated. Let's ignore it.

I'm curious to know what changes there are between running as headless true vs false.

@optikalefx The major change is a user agent - chrome headless identifies itself as HeadlessChrome. Try running the following script in headless and headful modes:
const puppeteer = require('puppeteer');

(async() => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  console.log(await page.evaluate(() => navigator.userAgent));
  browser.close();
})();

User agent is sent with every request as a user-agent header. If there's a need, user-agent could be changed with the page.setUserAgent method.

In non-headless mode, screenshots work differently because my screen is in HiDPI mode (MacBook Retina). Here's one of the 'different' screenshots:

@LoganDark please, file a separate issue.

Here is a gist of the problem.

@rosshadden try overriding user-agent in your gist. If this doesn't help, please file a separate issue.

Yeah, about 3 times already I've been told to file a different issue.
Obviously I haven't. And I won't, at least not right now.
Stop telling me to.

@aslushnikov we need to re-open this ticket IMO. I'm sorry that this issue had unrelated things in it. Setting the user-agent doesn't change anything - as in something is still different about the request. The result of that user-agent log after it's set is exactly what I set it to.
Can you think of anything else that changes when headless is set to true? Something that Amazon is able to detect? Maybe something about cookies? Maybe you could guide me in the right direction in the code and I can look through myself. Being unfamiliar with the codebase would make having a quick guidance very helpful.

There are a few ways Amazon can be detecting headless access. Nothing can really be done internally about them if Amazon is implementing any techniques like this.
The only primary difference is the Headless in the UA string. Beyond that, everything should be functioning the same from the user perspective of headless, as stated before.

@Garbee super interesting. So, why can't we just define things like language, plugins etc? I can't set things on navigator, but I can polyfill other methods to prevent detection. Maybe you guys can set the navigator settings?

It looks like I can polyfill navigator using
Object.defineProperties(navigator, {
			  'plugins': {
			    value: ['adBlock'],
			    writable: true
			  }
			});

Well I polyfilled everything in that article, and it passes all of those tests after the goto statement. But it still is getting caught.  quite interesting.

@aslushnikov While my gist doesn't have a UA set, setting it was the first thing @optikalefx tried when we discovered this problem.  What I can do is update my gist with setting the UA and the polyfills/workarounds we have tried since.

@optikalefx @rosshadden Chrome headless is built atop of content/ layer and doesn't include chrome/ layer, whereas chrome headful includes both content/ and chrome/ layers. So naturally, there might be multiple subtle ways to detect headless.
More on chromium architecture could be found here:

Chromium: displaying a web page in chromium
Chromium: content module

I also have the same problem using the option "headless:true", the site I have scraped is https://mobile.bet365.com, and it only works using the option "headless:false" otherwise it emits the following message. error:"""
[0418/092440.728:INFO:CONSOLE(0)] "Mixed Content: The page at 'https://mobile.bet365.com/?apptype=&appversion=&cb=1512512693#type=Coupon;key=18-20448857-48-1-36-0-0-0-1-0-0-4305-0-0-1-0-0-0-0-0-1454;ip=0;lng=1;anim=1' was loaded over HTTPS, but requested an insecure XMLHttpRequest endpoint 'http://<IP_ADDRESS>/'. This request has been blocked; the content must be served over HTTPS.", source: https://mobile.bet365.com/?apptype=&appversion=&cb=1512512693#type=Coupon;key=18-20448857-48-1-36-0-0-0-1-0-0-4305-0-0-1-0-0-0-0-0-1454;ip=0;lng=1;anim=1 (0)

@koreus7 - Solution worked for Amazon issue reported by @optikalefx

Full code to Scrap Amazon behind Login Wall - Optimized and Works in Headless Mode (Avoid BOT detection)
// Get addressess from Amazon Address Book

const puppeteer = require('puppeteer');

(async () => {

  // Syntactic Sugar

  const Navigate = async (url) => {
    await page.goto(url);
  }

  const EnterText = async (selector, text) => {
    await page.click(selector);
    await page.keyboard.type(text);
  }

  const ClickNavigate = async (selector, waitFor = -1) => {
    await page.click(selector);
    if (waitFor >= 0) {
      await page.waitFor(waitFor*1000)
    }
    else {
      await page.waitForNavigation();
    }
  }

  // Main Flow

  const C_HEADELESS = true
  const C_OPTIMIZE = true
  const C_SLOWMOTION = 0 // slow down by X ms

  const browser = await puppeteer.launch({
    headless: C_HEADELESS,
    slowMo: C_SLOWMOTION
  });
  const page = await browser.newPage();

  // To ensure Amazon doesn't detect it as a Bot
  await page.setExtraHTTPHeaders({
    'Accept-Language': 'en-GB,en-US;q=0.9,en;q=0.8'
  });

  // No unwanted resources
  if (C_OPTIMIZE) {
    await page.setRequestInterception(true);
    const block_ressources = ['image', 'stylesheet', 'media', 'font', 'texttrack', 'object', 'beacon', 'csp_report', 'imageset'];
    page.on('request', request => {
      //if (request.resourceType() === 'image')
      if (block_ressources.indexOf(request.resourceType) > 0)
        request.abort();
      else
        request.continue();
    });
  }

  // Creds
  const USER_EMAIL = "YOUR_EMAIL_HERE"
  const USER_PASSWORD = "YOUR_PASSWORD_HERE"

  // Home Page constants
  const U_HOMEPAGE = 'https://amazon.com'
  const U_LOGIN_PAGE = 'https://www.amazon.com/ap/signin?clientContext=135-8638983-8261231&openid.return_to=https%3A%2F%2Fwww.amazon.com%2Fa%2Faddresses&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.assoc_handle=usflex&openid.mode=checkid_setup&marketPlaceId=ATVPDKIKX0DER&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&pageId=usflex&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&openid.pape.max_auth_age=900&siteState=clientContext%3D143-3525329-4850620%2CsourceUrl%3Dhttps%253A%252F%252Fwww.amazon.com%252Fa%252Faddresses%2Csignature%3Dnull'
  const S_LOGIN_LINK = '#nav-link-accountList'

  // Optimzed the flow to reach address book faster, trick is to manually try to go to Target page before login and will be hit
  // by the Amazon Login Wall, capture the URL which will now have return page set to openid.return_to field in the url
  // This helps to land on the target page direclty after login without having to browse through heavy Home page
  // Caution: Trying to go to Address Book directly (any page with sensitive information) will challenge the user with additional password screen.

  // Commented, since this is now optimized
  // ------------------------------------------
  // // Go to Home Page
  // await Navigate(U_HOMEPAGE)
  //
  // // Go to Login Page
  // await ClickNavigate(S_LOGIN_LINK, 1)
  // ------------------------------------------

  // Go directly to Login Page
  await Navigate(U_LOGIN_PAGE) // USER-ACTION

  // Login Page constants
  const S_EMAIL_TEXT = '#ap_email'
  const S_CONTINUE_BUTTON = '#continue'
  const S_PASSWORD_TEXT = '#ap_password'
  const S_SIGNIN_BUTTON = '#signInSubmit'

  // Login - Step 1
  await EnterText(S_EMAIL_TEXT, USER_EMAIL); // USER-ACTION
  await ClickNavigate(S_CONTINUE_BUTTON); // USER-ACTION

  // Login - Step 2
  await EnterText(S_PASSWORD_TEXT, USER_PASSWORD); // USER-ACTION
  await ClickNavigate(S_SIGNIN_BUTTON); // USER-ACTION

  // Enter password again - Secondary Protection - This is required only if you try to land on the page with sensitive information directly
  await EnterText(S_PASSWORD_TEXT, USER_PASSWORD); // USER-ACTION
  await ClickNavigate(S_SIGNIN_BUTTON); // USER-ACTION

  // AddressBook constants
  const U_ADDRESSBOOK = 'https://www.amazon.com/a/addresses'
  const S_ADDRESS_TILE = '.normal-desktop-address-tile'

  const S_ADDRESS_FULLNAME = '#address-ui-widgets-FullName'
  const S_ADDRESS_LINEONE = '#address-ui-widgets-AddressLineOne'
  const S_ADDRESS_LINETWO = '#address-ui-widgets-AddressLineTwo'
  const S_ADDRESS_CITYSTATEPOSTALCODE ='#address-ui-widgets-CityStatePostalCode'
  const S_ADDRESS_COUNTRY = '#address-ui-widgets-Country'
  const S_ADDRESS_PHONENUMBER = '#address-ui-widgets-PhoneNumber'
  const S_ADDRESS_NODEFAULT = '.address-section-no-default'
  const S_ADDRESS_DEFAULT = '.default-section'
  const S_ADDRESS_DEFAULT_FRESH = '#ya-myab-fresh-address-icon'
  const S_ADDRESS_DEFAULT_AMAZON = '#ya-myab-default-shipping-address-icon'

  // Commented, since this is now optimized
  // ------------------------------------------
  // // Go to AddressBook
  // await Navigate(U_ADDRESSBOOK)
  // await page.screenshot({path: '5.png'});
  // ------------------------------------------

  // Get All Addresses
  const allAddressElements = await page.$$(S_ADDRESS_TILE);

  const getAddresses = allAddressElements.map(async (addressElement) => {

    let defaultAddressforAmazon = false
    let defaultAddressforFresh = false

    const defaultAddressElement = await addressElement.$(S_ADDRESS_DEFAULT)
    if (defaultAddressElement !== null) {
      const defaultAddressForAmazonElement = await defaultAddressElement.$(S_ADDRESS_DEFAULT_AMAZON)
      defaultAddressforAmazon = defaultAddressForAmazonElement ? true: false

      const defaultAddressForFreshElement = await defaultAddressElement.$(S_ADDRESS_DEFAULT_FRESH)
      defaultAddressforFresh = defaultAddressForFreshElement ? true: false
    }

    const fullNameElement = await addressElement.$(S_ADDRESS_FULLNAME)
    const fullName = await (await fullNameElement.getProperty('innerHTML')).jsonValue();

    const addressLineOneElement = await addressElement.$(S_ADDRESS_LINEONE)
    const addressLineOne = await (await addressLineOneElement.getProperty('innerHTML')).jsonValue();

    const addressLineTwoElement = await addressElement.$(S_ADDRESS_LINETWO)
    const addressLineTwo = addressLineTwoElement ? await (await addressLineTwoElement.getProperty('innerHTML')).jsonValue() : '';

    const cityStatePostalCodeElement = await addressElement.$(S_ADDRESS_CITYSTATEPOSTALCODE)
    const cityStatePostalCode = await (await cityStatePostalCodeElement.getProperty('innerHTML')).jsonValue();

    const countryElement = await addressElement.$(S_ADDRESS_COUNTRY)
    const country = await (await countryElement.getProperty('innerHTML')).jsonValue();

    const phoneNumberElement = await addressElement.$(S_ADDRESS_PHONENUMBER)
    let phoneNumber = await (await phoneNumberElement.getProperty('innerHTML')).jsonValue();
    phoneNumber = phoneNumber.split(':')
    phoneNumber = phoneNumber[1].trim()

    return {
      FullName: fullName,
      AddressLineOne: addressLineOne,
      AddressLineTwo: addressLineTwo,
      CityStatePostalCode: cityStatePostalCode,
      Country: country,
      PhoneNumber: phoneNumber,
      DefaultAddressforAmazon: defaultAddressforAmazon,
      DefaultAddressforFresh: defaultAddressforFresh
    }

  });

  let addresses = await Promise.all(getAddresses)
  console.log(addresses)
  await browser.close();
})();

This is an absolute pearl. Thanks for sharing the code above.

For what it's worth I've also found that adding the following user agents override can help smooth over differences in some cases:
await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36')

The UA I've provided is just an example. You can use any valid UA that matches an existing browser.

as @jondlm said, UserAgent option make headless selenium work do same with non-headless selenium. thx.

@koreus7 setting the languages works like a charm!

I get it works by adding this 2
await page.setExtraHTTPHeaders({
    'Accept-Language': 'en-US,en;q=0.9'
});
await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36');

Must thanks to @koreus7 & @jondlm , it won't if miss out any 1 of it.
P/S: I was trying to access this site www.blibli.com

I've made a fake user agent generator that works pretty fine!
function* generateUserAgent() {
  let webkitVersion = 10;
  let chromeVersion = 1000;

  const so = [
    'Windows NT 6.1; WOW64',
    'Windows NT 6.2; Win64; x64',
    "Windows NT 5.1; Win64; x64",
    'Macintosh; Intel Mac OS X 10_12_6',
    "X11; Linux x86_64",
    "X11; Linux armv7l"
  ];
  let soIndex = Math.floor(Math.random() * so.length);

  while (true) {
    yield `Mozilla/5.0 (${so[soIndex++ % so.length]}) AppleWebKit/537.${webkitVersion} (KHTML, like Gecko) Chrome/56.0.${chromeVersion}.87 Safari/537.${webkitVersion} OPR/43.0.2442.991`;

    webkitVersion++;
    chromeVersion++;
  }
}

const userAgents = generateUserAgent();

// ...
await page.setUserAgent(userAgents.next().value);

So headless true/false change user agent and other stuffs?
i have two different test that works on headless:false mode but fails on headless:true mode due to rendering differences of fonts and due to time needed to make a button clickable, but i cannot share due to confindential website.
I think headless true/false should not change rendering process.
Should i consider to set a common user agent to make behaviour more consistent?
thanks.

My case is completely the opposite of the OP's situation. I got an Amazon's robot check while headless mode:false, and bypass while headless mode:true. I solved this issue thanks to @koreus7 Many thanks 👍

Using @koreus7  and @jondlm  comments solved my problem

Recently, I had the same experience of getting blocked because of using headless browser. While scraping a popular website.  Even after adding proper headers and user agent it didn't work out.
Finally used puppeteer-extra with stealth mode plugin  which fixed the problem.
This thread helped me  a lot to figure out what all could go wrong.
Thanks @Garbee  @optikalefx

@Bhabaranjan19966  so this https://github.com/berstend/puppeteer-extra/tree/master/packages/puppeteer-extra with this https://www.npmjs.com/package/puppeteer-extra-plugin-stealth ? i will try, thanks.

not working for me : headless and gui mode renders page in a little different way

@Bhabaranjan19966 so this https://github.com/berstend/puppeteer-extra/tree/master/packages/puppeteer-extra with this https://www.npmjs.com/package/puppeteer-extra-plugin-stealth ? i will try, thanks.

yes those are the two repositories, fixed my problem  of hiding headless identity. At least for the web page I was trying to scrap. @andreabisello

@Bhabaranjan19966 so this https://github.com/berstend/puppeteer-extra/tree/master/packages/puppeteer-extra with this https://www.npmjs.com/package/puppeteer-extra-plugin-stealth ? i will try, thanks.

Yes, those are the two repositories fixed my problem. @andreabisello

I'm having this same issue with peapod.com right now. In headful mode, my program runs successfully. In headless mode, I'm screenshotting to debug and see that the link is clicked, spinner is activated, but the page never changes. How can I debug this better? @aslushnikov , could you provide me some guidance?

Recently, I had the same experience of getting blocked because of using headless browser. While scraping a popular website. Even after adding proper headers and user agent it didn't work out.
Finally used puppeteer-extra with stealth mode plugin which fixed the problem.
This thread helped me a lot to figure out what all could go wrong.
Thanks @Garbee @optikalefx

The stealth mode did the trick for me too! TYVM

None of these suggested solutions work on Mac OS X. To reproduce:

Change your system language to something other than en-US or en, so that applications use that locale.
Test a browser extension or web site that is internationalised by selected user locale.
It is impossible to test or change the browser locale to en-US on non-headless mode at least.

For what it's worth I've also found that adding the following user agents override can help smooth over differences in some cases:
await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36')

The UA I've provided is just an example. You can use any valid UA that matches an existing browser.

Add this just below where page is defined.

In my case if I login via SSH and run the same code via SSH it doesn't work.
If I login via GUI then run terminal in gui, it works...
Same code, headless is true...

I'm curious to know what changes there are between running as headless true vs false. When I run a login to Amazon using headless: true I get an error from Amazon via the screenshot. But when I set headless: false I watch it work just fine, no error.
So I'm trying to figure out what headless: true is doing that is different from when it's not headless.
Thanks to any suggestions.

I am using puppeteer to make an simple automation script to login into my google account, it's working fine in headless: false mode but in case of headless: true it's showing selector not found;

Can anyone provide an actual example file to run that reproduces this issue?

i have the same problem on grainger.com. When set --headless : false it is working but headless: true return promise handling error
any help will be appreciated

All the above methods are not working for me, I am still facing issues related to headless and normal mode. Any help will be appreciated

I got same issue and with combination of puppeteer-extra and following lib solved the issue
https://www.npmjs.com/package/puppeteer-extra-plugin-stealth

@Garbee Here is the code. This happens only for localhost if I give the actual website URL (http://www.google.com... etc) it is working for both options.
const browser = await puppeteer.launch({headless: true});
  const page = await browser.newPage();
  await page.goto('localhost:3000', {
          networkIdleTimeout: 1000,
          waitUntil: 'networkidle',
          timeout: 3000000
        });
  console.log(await page.url());

Output: about:blank
Expected output: localhost:3000
If headless is false I am getting the expected output.

For what it's worth I've also found that adding the following user agents override can help smooth over differences in some cases:
await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36')

The UA I've provided is just an example. You can use any valid UA that matches an existing browser.

bro it's working

Recently, I had the same experience of getting blocked because of using headless browser. While scraping a popular website. Even after adding proper headers and user agent it didn't work out.
Finally used puppeteer-extra with stealth mode plugin which fixed the problem.
This thread helped me a lot to figure out what all could go wrong.
Thanks @Garbee @optikalefx

Thanks Man Saved life


add a load_model function and use it to allow loading multiple versions of models simultaneously

This is a proof of concept for a slightly more structured model loading than the
current one which is based on modifying the LOAD_PATH and loading the module
as a package. The issue with the current system is that you can not load
multiple models with the same name since the Julia namespace for files loaded as
packages is just the name of the module. However, using the Julia package system
for the models means that it can be precompiled which can generate big time
savings over multiple sessions where the same model is loaded.
This PR creates a new load_model function with the intention of keeping the
advantages of precompiling the models which also allow loading multiple models
of the same name. This is done by having an optional argument to load_model
that can give a separate "alias" for the model. When this is given, the module Model part in the model is replaced with the alias. The model file is copied
to a scratch space before getting loaded in order to not pollute the folder where
they are loaded from.
The drawbacks to this approach are:

Splitting a model into multiple files with include will not work out of the box.
Stacktraces etc will also point to this generated file in the scratch space

Below is a usage example:
[ Info: Precompiling simple_RBC [top-level]
6 variable(s): C, K, L, w, r, A
1 shock(s): ea
7 parameter(s): g = 0.015, α = 0.33, γ = 0
    δ = 0.1, λ = 0.97, ρ = 0.03, β = @link 1 / (1 + ρ)
6 equations(s):
   E1:   @log 1 / C[t] = β * (1 / (C[t + 1] * (1 + g))) * ((r[t + 1] + 1) - δ)
   ...

In [3]: RBC2 = load_model("../2.simple_RBC/simple_RBC.jl", "simple_RBC2")
[ Info: Precompiling simple_RBC2 [top-level]
6 variable(s): C, K, L, w, r, A
1 shock(s): ea
7 parameter(s): g = 0.015, α = 0.33, γ = 0
    δ = 0.1, λ = 0.97, ρ = 0.03, β = @link 1 / (1 + ρ)
6 equations(s):
   E1:   @log 1 / C[t] = β * (1 / (C[t + 1] * (1 + g))) * ((r[t + 1] + 1) - δ)
...

Since this is the first time the model is loaded it has to be precompiled.
And a model with a different alias also need to be precompiled.
In a follow up session, the models are already precompiled and load quickly:
julia> RBC = load_model("../2.simple_RBC/simple_RBC.jl");

julia> RBC2 = load_model("../2.simple_RBC/simple_RBC.jl", "simple_RBC2");

Codecov Report

Merging #45 (e1cad21) into master (9fee01f) will decrease coverage by 1.22%.
The diff coverage is 8.33%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##           master      #45      +/-   ##
==========================================
- Coverage   94.45%   93.23%   -1.22%
==========================================
  Files          15       15
  Lines        1676     1700      +24
==========================================
+ Hits         1583     1585       +2
- Misses         93      115      +22

Impacted Files
Coverage Δ

src/ModelBaseEcon.jl
100.00% <ø> (ø)

src/model.jl
92.83% <8.33%> (-3.80%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


DataGrid: Could not resolve type DataGridCell

I am using Avalonia and DataGrid (0.8.0). Including DataGrid theme into the application provokes the following error:
Error loading xaml at resm:Avalonia.Controls.DataGrid.Themes.Default.xaml?assembly=Avalonia.Controls.DataGrid: Could not convert object 'DataGridCell' (of type System.String) to {clr-namespace:Avalonia.Styling;assembly=Avalonia.Styling}Selector: Could not resolve type 'DataGridCell'

.csproj
    <ItemGroup>
        <PackageReference Include="Avalonia" Version="0.8.0" />
        <PackageReference Include="Avalonia.Controls.DataGrid" Version="0.8.0" />
    </ItemGroup>

App.xaml
<Application xmlns="https://github.com/avaloniaui">
    <Application.Styles>
        <StyleInclude Source="avares://Avalonia.Themes.Default/DefaultTheme.xaml" />
        <StyleInclude Source="avares://Avalonia.Themes.Default/Accents/BaseLight.xaml" />
        <StyleInclude Source="resm:Avalonia.Controls.DataGrid.Themes.Default.xaml?assembly=Avalonia.Controls.DataGrid" />
    </Application.Styles>
</Application>

The following workaround exists: execute some code to make sure Avalonia.Controls.DataGrid is loaded before using AvaloniaXamlLoader.
App.xaml.cs
public override void Initialize()
{
    var dataGridType = typeof(DataGrid); // HACK
    AvaloniaXamlLoader.Load(this);
}

Yes, there is currently a problem with portable.xaml in that it doesn't load all referenced assemblies correctly.
Your workaround works, but we've also added a UseDataGrid extension method for AppBuilder which will do the same thing in a slightly tidier way.

Okay, I didn't know about this extension method. Should we leave this issue opened?

Yeah, DataGrid has no documentation yet, so lets leave this open for other people looking for the solution.

The UseDataGrid method is now no longer needed since #2322 so closing.

I am using Avalonia and DataGrid (0.8.0). Including DataGrid theme into the application provokes the following error:
Error loading xaml at resm:Avalonia.Controls.DataGrid.Themes.Default.xaml?assembly=Avalonia.Controls.DataGrid: Could not convert object 'DataGridCell' (of type System.String) to {clr-namespace:Avalonia.Styling;assembly=Avalonia.Styling}Selector: Could not resolve type 'DataGridCell'

.csproj
    <ItemGroup>
        <PackageReference Include="Avalonia" Version="0.8.0" />
        <PackageReference Include="Avalonia.Controls.DataGrid" Version="0.8.0" />
    </ItemGroup>

App.xaml
<Application xmlns="https://github.com/avaloniaui">
    <Application.Styles>
        <StyleInclude Source="avares://Avalonia.Themes.Default/DefaultTheme.xaml" />
        <StyleInclude Source="avares://Avalonia.Themes.Default/Accents/BaseLight.xaml" />
        <StyleInclude Source="resm:Avalonia.Controls.DataGrid.Themes.Default.xaml?assembly=Avalonia.Controls.DataGrid" />
    </Application.Styles>
</Application>

The following workaround exists: execute some code to make sure Avalonia.Controls.DataGrid is loaded before using AvaloniaXamlLoader.
App.xaml.cs
public override void Initialize()
{
    var dataGridType = typeof(DataGrid); // HACK
    AvaloniaXamlLoader.Load(this);
}

Also works with  Avalonia.Svg.Skia.Svg for

XamlX.XamlParseException: Unable to resolve type SvgImage from namespace https://github.com/avaloniaui Line 17, position 16.


LIVE-2214 - Disable uppercase when displaying an address in DeviceAction

In buy flow (an possibly in other flows), the user has to check that the address displayed on device matches the one displayed in-app. The issue is that in the app it's written with uppercase.
With this fix we remove the "uppercase" text transformation that was applied in the component displaying the address, so the address will be displayed with its original casing.
Before

After

Type
Bug fix
Context
LIVE-2214
Parts of the app affected / Test plan

tested on Android (samsung) and everything looks fine 👍


Add a condition to the BuildWebAssets

What's the intended benefit of having the condition?
Do we need to update our Dockerfile (or CI/CD pipeline etc.) to incorporate this into our deployment build now?


Push December 2014 release of Public Safety Incident Mapping to codebase

Grab the December 2014 release of Public Safety Incident Mapping and ensure the most up to date codebase is in github

https://github.com/Esri/incident-import-tools/commit/0f76b5d5db060bce25e7b6fd0fbcd482aefea385
Checked in


springboot 2.0.0   druid-spring-boot-starter 1.1.9 不能启动

20-Mar-2018 00:13:46.180 警告 [RMI TCP Connection(2)-<IP_ADDRESS>] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [demoTest] registered the JDBC driver [com.alibaba.druid.proxy.DruidDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
20-Mar-2018 00:13:46.181 警告 [RMI TCP Connection(2)-<IP_ADDRESS>] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [demoTest] registered the JDBC driver [com.mysql.cj.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.

你这个是停止程序的时候报的内存泄漏警告吧？


I dont understand

When i open the program. Its shows me a Failure and tells me:
Vulkan drivers installed and funcotioning but are incompatible. An instace could not be created.
So i dont understand what it means.

It just means vkCreateInstance() failed. The program doesn't actually
investigate as to why it failed and just prints that error. Either I
made a mistake in one of the parameters when creating a Vulkan instance
(likely) or your Vulkan driver is faulty.

I think i dont have a vulkan driver.


Review contributor journey

https://miro.com/app/board/o9J_kwGZZ7U=/

[ ] Possibility of usability testing the workflow

The current contributor workflow looks as follows:

contributor forks the dutyfree repo.
They add their resource (to either a separate file or recources.yml)
They create a pull request against the repo
maybe: a pipeline validates the resource (dockerhub pullable and runable)
Concourse maintainers to accept the pull request
after the PR is merged in, a pipeline pushes the new resource into the dutyfree db (using either a cli tool or the server's api).
Since the server reads directly from the db, that will result into an instantaneous update for dutyfree.

stories: #139 #140 #141 will be capturing in more details the design and development of the above workflow.


Pre/Post Deployment script - Script does not Starts the Trigger post deployment

The Pre/Post deployment works fine, it Stops the Trigger as Expected but once down

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1e22e4d6-366f-fbf8-e141-8905ca7ab7ae
Version Independent ID: 5be2acf2-66e7-cadf-6666-5a85a65b9771
Content: Continuous integration and delivery in Azure Data Factory - Azure Data Factory
Content Source: articles/data-factory/continuous-integration-deployment.md
Service: data-factory
GitHub Login: @djpmsft
Microsoft Alias: daperlov

Script does not Start the Trigger post deployment, if opted not to delete the Deployment
$deleteDeployment=$false (in post deployment script)

@chandra9787 Thank you for your feedback. We will investigate and update the thread.

@djpmsft Can you please take a look at this?

The $deleteDeployment does not affect the triggering. Are your triggers started in the ARM template? The deployment should reflect the status in the template.
Best,
Daniel

Hi Daniel,
My apologies, the triggers were started in ADF but for some reason the build was stick with old ARM master template were the triggers were stopped. So, it’s totally working fine, I missed to update in issue.
Thanks,
Chandra
Sent from my iPhone
On Jul 10, 2020, at 10:45 PM, Daniel Perlovsky<EMAIL_ADDRESS>wrote:

The $deleteDeployment does not affect the triggering. Are your triggers started in the ARM template? The deployment should reflect the status in the template.
Best,
Daniel
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/MicrosoftDocs/azure-docs/issues/57673#issuecomment-656787583, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AG6NMZDXWEEQSNZMBONWHHLR25EB5ANCNFSM4OFSXDLA.

Glad you are unblocked! I'll go ahead and close the issue then
#please-close


DOC: improve signal.dbode docstring.

Unlike dfreqresp, which from context makes it clear what "radian/samples" means, dbode lacks such an example. This can be helpful for non-native speakers.
The unit for the returned frequency array w was wrong, since dt is specified to be in seconds.
[docs only]

Thank you for your PR @ericonr . Indeed it reads better to me.
For your second remark, that's not necessarily in seconds since the model can be in any time unit. You can still have an LTI model for millisecond dynamics which is typically done for fast changing systems to avoid working with huge numbers. That's why it says time unit whatever you modeled in the first place.

@ilayn that's fair. But the docstrings for the three types of models mention seconds explicitly, so someone who deviates from that when defining their model will know the result from dbode will be in a different unit as well, no? And time_unit isn't used anywhere else, so for consistency I thought seconds made more sense.
https://github.com/scipy/scipy/blob/d8f96860461718355249fe9db66e2f6d16b4a480/scipy/signal/_ltisys.py#L513-L516
If you disagree, I can remove that change from my PR no problem.
Maybe the best change would be clarifying that dt can be seconds or time_unit, whatever that is. Do you think it would be a good addition to the docs? I like it, but it feels like it might add confusion for some.

Gentle ping at @ericonr: Are you still interested in finishing this PR or do you want me to complete if for you?

Hi @DietBru, sorry for the delay, I got a bit caught up and didn't manage to respond to your review
I think there was some confusion on the concepts involved in these functions, though.

I took the liberty to revise this PR a bit, so it will hopefully not be too late for release 1.15.

The rendered result can be found here.

Fyi 1.15.0 has now been branched:)

And thanks for your first contribution to SciPy @ericonr keep them coming!

Fyi 1.15.0 has now been branched:)

Dang. I hoped that the due date of 2025-01-01 on the milestone label was authoritative...
Nonetheless, thanks for your contribution, @ericonr and thanks for the support @j-bowhay, @ilayn @WarrenWeckesser!

I hoped that the due date of 2025-01-01 on the milestone label was authoritative...

I think it could be backported still, but that's at @tylerjereddy's discretion (and usually for bug fixes)

Looks like pure docstring so I'll put the backport label for consideration. Release schedule discussion is here: https://discuss.scientific-python.org/t/proposed-release-schedule-for-scipy-1-15-0/1502
Indeed, as Lucas says, we usually just backport bug fixes.

Thanks for considering, @tylerjereddy. I am perfectly  fine if this PR doesn't get backported. The primary aim of my comment was just to highlight the date discrepancy of the milestone deadline.

Thank you! I wouldn't have had the time until later this week, at the earliest :)


Omnisharp crashes way too easily

If there is a bad solution in the folder that omnisharp tries to load, it crashes and does not allow switching to the correct solution (and you are usually forced to delete the bad solution file).
If one of the projects in the solution is invalid, omnisharp crashes once again and fails to load valid projects.

@cra0zy Can you share an example omnisharp log?

This repository shows both problems: https://github.com/picoe/Eto
It will try to load bad solution at start, and bottom left you can't switch to the correct solution (however what I've discovered earlier today is that you do have an option to switch the project using command palette, its just that bottom left corner gets bugged...).
In case of the second part, try opening Eto.sln, the first project it fails to load will make it stop loading completely.

Does OmniSharp actually crash here? Or is it just that it fails to load the project?


Titles should start with "Step" and not end with "."

The messages in the progress bar as well as the step references in the report should be consistent format:

Step 6.X.Y Action performing
Start 6.X.Y Title

Failure and warning messages should not include "Step"

Desired changes will be provided via email directly.  Abridged examples are shown below.
From the detail report section
Part 1 KOEO Data Collection............................................(FAIL)
Test 1.1. Test vehicle data collection.................................(PASS)
Test 1.2. Verify engine operation......................................(PASS)
Test 1.3. DM5: Diagnostic readiness 1..................................(PASS)
Test 1.4. DM24: SPN support............................................(PASS)
From the test log report
Start Test 1.1. Test vehicle data collection
...
End Test 1.1. Test vehicle data collection
Start Test 1.2. Verify engine operation
...
End Test 1.2. Verify engine operation
Start Test 1.3. DM5: Diagnostic readiness 1
...
End Test 1.3. DM5: Diagnostic readiness 1
Start Test 1.4. DM24: SPN support


[DOCFIX] Update user-configuration.yml of Chinese version

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/12321/
Test PASSed.


OutputStream::try_default: Fallback to non-default devices

This pr enhances OutputStream::try_default so that it can fallback to non-default devices on failure.
I'm testing this change to help address a Linux error that was reported for Robo Instructus, so hopefully I'll be able to report back if this change helped.
The device doesn't support any format!?: BackendSpecific { err: BackendSpecificError { description: "ALSA function \'snd_pcm_open\' failed with error \'EACCES: Permission denied\'" } } -    0: robo_instructus_compute::error::panic_hook::{{closure}}
1: std::panicking::rust_panic_with_hook
          at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:597:17
2: std::panicking::begin_panic_handler::{{closure}}
          at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:499:13
3: std::sys_common::backtrace::__rust_end_short_backtrace
          at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:141:18
4: rust_begin_unwind
          at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:495:5
5: core::panicking::panic_fmt
          at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/core/src/panicking.rs:92:14
6: core::option::expect_none_failed
          at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/core/src/option.rs:1268:5
7: <cpal::platform::platform_impl::Device as rodio::stream::CpalDeviceExt>::new_output_stream
8: rodio::stream::OutputStream::try_default

Looks like this change did indeed address the issue with multiple output devices being available in one Linux setup. This allowed the sound to work, presumably after failing the default choice. So I think it's worth having in rodio.
In case there's interest heres the output, presumably fail logs when selecting the devices that didn't work
ALSA lib pcm_dmix.c:1038:(snd_pcm_dmix_open) unable to create IPC semaphore
ALSA lib pcm_dmix.c:1038:(snd_pcm_dmix_open) unable to create IPC semaphore
ALSA lib pcm_dsnoop.c:638:(snd_pcm_dsnoop_open) unable to open slave
ALSA lib pcm_dmix.c:1038:(snd_pcm_dmix_open) unable to create IPC semaphore
ALSA lib pcm_dsnoop.c:638:(snd_pcm_dsnoop_open) unable to open slave
connect(2) call to /dev/shm/jack-1001/default/jack_0 failed (err=No such file or directory)
attempt to connect to server failed
connect(2) call to /dev/shm/jack-1001/default/jack_0 failed (err=No such file or directory)
attempt to connect to server failed
ALSA lib pcm_oss.c:377:(_snd_pcm_oss_open) Unknown field port
ALSA lib pcm_oss.c:377:(_snd_pcm_oss_open) Unknown field port


Window/Input Manager

[x] Create windows
[ ] Input Bindings
Hardware that is connected to specific actions (WASD, Spacebar, left joystick, etc.)
[ ] Input Actions
Specific actions that result from a player's input (move, jump, etc.)

Can this be closed @joaopat98 @DannyIsYog?

No, we still have no input manager, just window events.

Lambda functions don't work when using them as callback on the register binding functions, let it be given by us or the programmer. Need to find a work around that.


when I run the examples\server.js,throw a exception

`C:\Users\mark\Desktop\koa-restql-master\test\lib\prepare.js:55
let {
^
SyntaxError: Unexpected token {
`

It sounds like problem of node version. We only support node v6 at the time.
Besides, We only tested the package on Unix (no test on Windows).

You can try Windows Subsystem for Linux (beta) if you develop on Windows 10.

thanks, it's the problem of node version,my node version is v5.0,and then I update to the v6,and start the server,it's ok!

I think you can add the runtime environment needed，that's better

#9
I have edited README.

Yesterday, I actually have used babel turn your code, it prompts this error

is the windows system problem?

That IS Windows problem...
The best way to avoid this kind of problems is to use Windows Subsystem for Linux since we do not have any plan to solve this kind of problems.


Update Lemongrab Ult sounds

Way back they used to have double sounds for each unacceptable level, might look sketchy contributing like this but oh well

Unless you were referring to an earlier version of the original game, this doesn't appear to be accurate. Based on the latest version, the current implementation remains consistent with the intended design. Adding additional voicelines for Lemongrab, especially noisy ones, could potentially have a negative impact on the player experience


[4.0][BUG] Change Route Key to something besides ID

Bug report
When changing route key by overriding getRouteKeyName() as laravel docs read to change the url key backpack does not comply with the change
What I did
Override
public function getKeyName()
{
return 'uuid';
}

What I expected to happen
I expected the url to be of format /user/{uuid}/edit as opposed to /user/{id}/edit
What happened
no change in operation
What I've already tried to fix it
Made the following changes to
Backpack\CRUD\app\Library\CrudPanel\Traits\Read
public function getCurrentEntryId()
    {
        if ($this->entry) {
            //return $this->entry->getKey();
             return $this->entry->getRouteKey();
        }
.......................

public function getEntry($id)
    {
        if (! $this->entry) {
            //$this->entry = $this->model->findOrFail($id);
            $this->entry = $this->model->where($this->model->getRouteKeyName(),$id)->firstOrFail();
            $this->entry = $this->entry->withFakes();
        }

        return $this->entry;
    }

For this to work all references in blade files that reference
$entry->getKey();
//Need to be changed to
$entry->getRouteKey();

I have tested this breifly and I do not see any errors.
Backpack, Laravel, PHP, DB version
Laravel - 6.5.0
PHP 7.2.21
Backpack 4.0.12
When I run php artisan backpack:version the output is:

Also note that the delete trait also needs changed
Backpack\CRUD\app\Library\CrudPanel\Traits\Delete
public function delete($id)
    {
        //return (string) $this->model->findOrFail($id)->delete();
        return (string) $this->model->where($this->model->getRouteKeyName(),$id)->firstOrFail()->delete();
    }

As Well for update
Backpack\CRUD\app\Library\CrudPanel\Traits\Update
public function update($id, $data)
    {
        $data = $this->decodeJsonCastedAttributes($data);
        $data = $this->compactFakeFields($data);
       // $item = $this->model->findOrFail($id);
        $item = $this->model->where($this->model->getRouteKeyName(),$id)->firstOrFail();

        $this->createRelations($item, $data);

        // omit the n-n relationships when updating the eloquent item
        $nn_relationships = array_pluck($this->getRelationFieldsWithPivot(), 'name');
        $data = array_except($data, $nn_relationships);
        $updated = $item->update($data);

        return $item;
    }

Hello @MasterBratac
Thanks for submitting this issue.
As you can see this is something that changes some core functions of CRUD, and i am not sure that it only needs this small changes.
We need to create a proper PR and test everything beforehand.
I will label and tag this issue.
IF in the meanwhile you got yourself time and want to submit the PR i will happily test it :)
Best,
Pedro

@pxpm I will try and get that done this weekend so it can be tested.Thanks.

Thanks @MasterBratac for brought up this issue. Indeed, the core package itself should use $entry->getRouteKey() instead of $entry->getKey().
Backpack CRUD mainly deals with frontend interactivity, which usually accesses models through on route resource. In any serious project, the usage of different route resource identifier (like uuid) is very common.
It's probably due to simplicity or unknown legacy issues, the package stop at the straightforward ->getKey() approach, but I really hope this could be tested and merged.

Hello @MasterBratac
I was having a second look at this issue, as we are using getKey() in backpack, you could set the $primaryKey in your model to uuid ? Not sure it will work, give it a try.
It also as the $keyType,  laravel doc
Don't forget cached routes and configs.
Let me know how it goes :+1:
Best,
Pedro

Hi guys. Thanks for bringing this up.
I haven't tested UUIDs on Backpack in a while, but last time I did, I was able to use them pretty easily, by specifying a custom $primaryKey and $increments false on the model, like @pxpm said. PLEASE NOTE! I said $primaryKey not getRouteKeyName() 😄 I think there are two scenarios here, which I've outlined below, best to talk about them separately.
I think I see why there's confusion here. Backpack doesn't use route model binding. You'll notice when you defined the route (if you manually did that), you're not plugging a Model into the CRUD route. You're just pointing it to a CrudController, and the Model is defined inside the CrudController. That's why overwriting getRouteKeyName() on your model does nothing - because the route model binding functionality isn't actually used by Backpack. We could use it, of course, but I don't see the benefit of it right now. I think it's better to define the model inside the controller, so that everything you need to define about your CRUD stays inside one file - the ProductCrudController for example. I'm open to using route model binding, if it adds any benefit, but I don't see any. Maybe I can be convinced otherwise.
Let's dig into this issue. Copy-paste from the Model class in Laravel:
    public function getKeyName()
    {
        return $this->primaryKey;
    }

    public function getKey()
    {
        return $this->getAttribute($this->getKeyName());
    }

    public function getRouteKey()
    {
        return $this->getAttribute($this->getRouteKeyName());
    }

    public function getRouteKeyName()
    {
        return $this->getKeyName();
    }

I think there are two scenarios here:
Scenario 1 (UUID == primary key)
In your model you've done:
$primaryKey = 'uuid';

Or
public function getKeyName() { return 'uuid'; }

I think this is the way MOST people use UUIDs. Once you have UUIDs on your Model, they're the primary key.  So getKey() and getRouteKey() will return the same thing. AFAIK, this scenario already works in Backpack. You just need to overwrite $primaryKey on your model to specify the UUID column.
So for this use case, it's no use changing getKey() to getRouteKey(). The return of those methods is exactly the same, all the time.
Scenario 2 (UUID + separate primary key)
In your model you've done:
public function getRouteKeyName() { return 'uuid'; }

Because you can't overwrite $primaryKey, because both of these are true:

you're using UUIDs as route keys;
you're using regular primary keys too (incremented integers);

This one is trickier. You're probably:

using UUIDs because you don't want people to be able to see/increment IDs in the URL;
still using INT keys too because you think it'll be slightly faster to do DB operations that way;

I suspect you guys are struggling here, in Scenario 2? If so, please tell me why you've chosen to have both UUIDs and INT primary keys. What do you see as the benefit of using both? Where would you like to use UUIDs, and where INT keys? Personally I've never used both, I don't see the point. If you're using UUIDs on your routes, you're making DB calls using them anyway. So maybe you're using UUIDs for user interactions, but INT keys in commands, crons, things like that when no user interaction is needed?
Cheers!

@tabacitu  yes scenario 2 is what I am doing. Exposing UUID in URL and then INT incrementing keys internally. I could be completely off base but the reason I am doing this ( or would like to ) is for loading relations. The speed difference between how laravel stores a UUID as a 32 varchar vs INT or BigINT.
This way the internals should be a lot faster. How much I do not know, I suppose that really depends on the amount of records. I have one table that will start with nearly 100k records in it and grow from there.(Vehicles)

Got it @MasterBratac , thank you! I've already created PR https://github.com/Laravel-Backpack/CRUD/pull/2314  making these changes everywhere. Let me know if I missed something, or if I've overdone it. Please help make the PR better - it's a HUGE change so we have to make sure we don't break stuff.
But I'll be honest with you - it might take a while for us to merge this into master. There are so many changes, in so many important files, that I can't be 100% confident this won't break people's apps. That's why I'd rather push this change when we launch 4.1 (most likely in February 2020), no sooner.
If you want to use this feature until then, one way is to use this PR's branch. So in your composer.json you should require backpack/crud version dev-support-uuids-plus-int-pks instead of 4.0.*, then run composer update. Let me know how that branch works for you, and if you find any bugs related to this.
Cheers!
PS. Closing the issue, let's move the conversation to PR https://github.com/Laravel-Backpack/CRUD/pull/2314


Timeline: Improve events grouping

Use 'source' field in SQL queries instead of serialized 'timeline_group' => ['by' => '...', 'field' => 'owner_id'] in PHP.
Update code with "TODO: Should be updated when GROUP BY source will be used in sub selects."

Now group by 'source' is used in Validation query.


fix(reminders): improve the reminders layout, fix #3144

Description
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Remove the • for every notification
Add the arrow icon to the every notification
Add some padding to every notification and add the hover effect.
Add "Close" button and make it postpone the notification to the 5th day of the month, if by that time the monthly report is still not yet submitted.

Fixes # (issue)
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] Any dependent changes have been merged and published in downstream modules

Task linked: CU-86c18wnh2 [FIX] Improve the reminders layout

:tada: This PR is included in version 3.5.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


feat(semantic): syntax error for delete object?.#a

closes #7603

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#7636  👈 (View in Graphite)
#7635
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Merge activity

Dec 4, 12:32 AM EST: The merge label '0-merge' was detected. This PR will be added to the Graphite merge queue once it meets the requirements.


Filebeat Chart

[x] Chart version not bumped (the versions are all bumped and released at the same time)
[x] README.md updated with any new values or changes
[x] Updated template tests in ${CHART}/tests/*.py
[x] Updated integration tests in ${CHART}/examples/*/test/goss.yaml

@Crazybus pushing what I have thus far. I think the actual body of the yaml templates, template tests, and docs are mostly complete, the remaining steps from my perspective are:

[ ] Adding goss testing to check for end-to-end "are filebeat indices being created" tests, which will involve releasing ES to a test cluster
[ ] Writing up accompanying CI definitions

I've pushed this to a branch on this repo instead of my fork so feel free to tinker with the branch if you want, I'm fine with either of us writing those tests and CI jobs, whoever gets to it first.
Implementation notes:

I left out PDBs since they don't make sense for a DaemonSet but left configurable tolerations in place, because they apply for a DaemonSet anyway and were in my copypasta :grin:
Old versions of the Chart used emptyDir, while the newer reference that the beats team maintains prefers hostPath, which I've reflected here. See the associated commit that made this change for additional information.
Do note that I'm using the internal monitoring HTTP port as a health check for the pod. I don't know whether I prefer this or using the config check -c flag, I leaned this way because it's a more definitive check in case the go process became unresponsive for whatever reason. The downside is that the docs mark this feature as experimental.
This is the first chart that will potentially manage RBAC resources. I didn't do anything crazy here aside from permitting the chart to flip these off with a boolean value.

Do note that I'm using the internal monitoring HTTP port as a health check for the pod. I don't know whether I prefer this or using the config check -c flag, I leaned this way because it's a more definitive check in case the go process became unresponsive for whatever reason. The downside is that the docs mark this feature as experimental.

This sounds good to me. I'm going to play with it a bit to see how it responds when the endpoint (such as elasticsearch) is down. It was only added in 6.4 but I don't see that being a problem as long as it is working in the latest 6.x and 7.x releases.


Update reference_policies_iam-condition-keys.md

Issue #, if available:
Description of changes:

Example for condition key iam:AssociatedResourceArn (works with ARN operators) should be using "ArnLike" (an ARN operator) instead of "StringLike" (not an ARN operator).
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Thanks for taking the time to submit a pull request.


Small typo fix in the README link of hosting

The nginx-example conf has a different name in the repo

Thanks!


writer: deallocate partially prepared statements on error

Summary
Writer does not call Close (and so that DEALLOCATE) if some of statements fail in the middle.
Although go's pg driver's Prepare is has map of prepared statements to prevent such errors (see the code excerpt below) it looks more correct to clean up in case of an error.
// Prepare creates a prepared statement with name and sql. sql can contain placeholders
// for bound parameters. These placeholders are referenced positional as $1, $2, etc.
//
// Prepare is idempotent; i.e. it is safe to call Prepare multiple times with the same
// name and sql arguments. This allows a code path to Prepare and Query/Exec without
// concern for if the statement has already been prepared.
func (c *Conn) Prepare(ctx context.Context, name, sql string) (sd *pgconn.StatementDescription, err error) {
	if name != "" {
		var ok bool
		if sd, ok = c.preparedStatements[name]; ok && sd.SQL == sql {
			return sd, nil
		}
	}
...
	sd, err = c.pgConn.Prepare(ctx, name, sql, nil)
...
	if name != "" {
		c.preparedStatements[name] = sd
	}

	return sd, nil
}

Test Plan
No test, it is hard/impossible to test b/c driver's Prepare caches statements.

@algorandskiy I suggest we ice box/close this for now - the driver for why the work was done may be resolved through configuration


BEP20 Token

Hi great work!
just curious is it possible to support BSC chain for this project?

Also i tried running yarn test but the tests don't run


Issue/fix duplicate in poetry lock file

Description
When I rebased these 3 dependabot bumps (452544a3a7f9f2ead874de58089461e0efa1f3c4 96a5b8418fce91762464afdf13436d8194ccd974 3a6382f6d6f93697fd17d967b98d052332529cf7) it resulted in some duplication in the poetry.lock file.
This PR fixes this issue.
Self Check:
Strike through any lines that are not applicable (~~line~~) then check the box

[ ] Attached issue to pull request
[ ] Changelog entry
[ ] Type annotations are present
[ ] Code is clear and sufficiently documented
[ ] No (preventable) type errors (check using make mypy or make mypy-diff)
[ ] Sufficient test cases (reproduces the bug/tests the requested feature)
[ ] Correct, in line with design
[ ] End user documentation is included or an issue is created for end-user documentation (add ref to issue here: )

Reviewer Checklist:

[ ] Sufficient test cases (reproduces the bug/tests the requested feature)
[ ] Code is clear and sufficiently documented
[ ] Correct, in line with design

Interesting. Perhaps we should use dependabot recreate then in the future.

Interesting. Perhaps we should use dependabot recreate then in the future.

Or maybe rebase more sequentially.


ROX-20542: disable http/2 in operator kube-rbac-proxy

Description
Cherry-picks https://github.com/stackrox/stackrox/pull/8407 into release-4.1
Checklist

[ ] Investigated and inspected CI test results
[ ] Unit test and regression tests added
[ ] Evaluated and added CHANGELOG entry if required
[ ] Determined and documented upgrade steps
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.
Testing Performed
Here I tell how I validated my change
TODO(replace-me)
Use this space to explain how you validated that your change functions exactly how you expect it.
Feel free to attach JSON snippets, curl commands, screenshots, etc. Apply a simple benchmark: would the information you
provided convince any reviewer or any external reader that you did enough to validate your change.
It is acceptable to assume trust and keep this section light, e.g. as a bullet-point list.
It is acceptable to skip testing in cases when CI is sufficient, or it's a markdown or code comment change only.
It is also acceptable to skip testing for changes that are too taxing to test before merging. In such case you are
responsible for the change after it gets merged which includes reverting, fixing, etc. Make sure you validate the change
ASAP after it gets merged or explain in PR when the validation will be performed.
Explain here why you skipped testing in case you did so.
Have you created automated tests for your change? Explain here which validation activities you did manually and why so.
Reminder for reviewers
In addition to reviewing code here, reviewers must also review testing and request further testing in case the
performed one does not seem sufficient. As a reviewer, you must not approve the change until you understand the
performed testing and you are satisfied with it.

/retest

CI looks good for release-4.1 branch.


Missing nodeSelector on challenge pods

I'm trying to get cert-manager working with Traefik in an AKS cluster running in Azure. I suppose this will work fine in an ordinary cluster - but our cluster has Windows nodes as well as Linux nodes.
And this doesn't work very well as Kubernetes tries to schedule cert-manager pods on a Windows host.
I have installed cert-manager using the current Helm chart. At first installation failed - but when I added three node selectors for the three pods, installation worked fine.
A node selector like
nodeSelector:
  beta.kubernetes.io/os: linux

is needed to ensure that pods are scheduled on a Linux node, not a Windows node.
But even though I've added this selector everywhere I can, there seems to be no way to add the same selector to the ACME challenge pods that cert-manager creates. They are created without selector - and my Kubernetes cluster then tries to schedule the pods on an idle Windows node.
Please add the capability to include a nodeSelector for the challenge pods as well.
/kind bug

To emphasize the problem: I've created a workaround for now by tainting all Windows nodes.
This works - but it's not the way to go. The proper solution would still be to add the right nodeSelector. I actually believe this can be hardcoded - cert-manager will always need a Linux node, won't it?

To emphasize the problem: I've created a workaround for now by tainting all Windows nodes.
This works - but it's not the way to go. The proper solution would still be to add the right nodeSelector. I actually believe this can be hardcoded - cert-manager will always need a Linux node, won't it?

I'm facing the exact same issue. Installed with helm chart v1.1.0 and set  nodeSelector: kubernetes.io/os: linux however this nodeselector does not get applied to the solver pods that are created for acme:
❯ kubectl get pods
NAME                                   READY   STATUS             RESTARTS   AGE
cm-acme-http-solver-kltk6              0/1     ImagePullBackOff   0          4m33s

❯ kubectl describe pod cm-acme-http-solver-kltk6
Name:         cm-acme-http-solver-kltk6
...
Node-Selectors:  <none>
...
Events:
  Type     Reason          Age                     From               Message
  ----     ------          ----                    ----               -------
  Normal   Scheduled       7m44s                   default-scheduler  Successfully assigned kubernetes-dashboard/cm-acme-http-solver-kltk6 to akswipool000000
  Normal   SandboxChanged  7m24s (x4 over 7m40s)   kubelet            Pod sandbox changed, it will be killed and re-created.
  Normal   Pulling         6m54s (x3 over 7m41s)   kubelet            Pulling image "quay.io/jetstack/cert-manager-acmesolver:v1.1.0"
  Warning  Failed          6m53s (x3 over 7m40s)   kubelet            Failed to pull image "quay.io/jetstack/cert-manager-acmesolver:v1.1.0": rpc error: code = Unknown desc = no matching manifest for windows/amd64 10.0.17763 in the manifest list entries
  Warning  Failed          2m40s (x24 over 7m36s)  kubelet            Error: ImagePullBackOff

So I get no matching manifest for windows/amd64 becuase it was assigned to a windows node.
The same issue existed in the chart for ingress-nginx but there they have now set it default to linux becuase it can only run on linux. I would agree that it could be the same here too unless cert-manager is planning to support running on windows nodes.

I'm facing the exact same issue. Installed with helm chart v1.1.0 and set  nodeSelector: kubernetes.io/os: linux however this nodeselector does not get applied to the solver pods that are created for acme:
❯ kubectl get pods
NAME                                   READY   STATUS             RESTARTS   AGE
cm-acme-http-solver-kltk6              0/1     ImagePullBackOff   0          4m33s

❯ kubectl describe pod cm-acme-http-solver-kltk6
Name:         cm-acme-http-solver-kltk6
...
Node-Selectors:  <none>
...
Events:
  Type     Reason          Age                     From               Message
  ----     ------          ----                    ----               -------
  Normal   Scheduled       7m44s                   default-scheduler  Successfully assigned kubernetes-dashboard/cm-acme-http-solver-kltk6 to akswipool000000
  Normal   SandboxChanged  7m24s (x4 over 7m40s)   kubelet            Pod sandbox changed, it will be killed and re-created.
  Normal   Pulling         6m54s (x3 over 7m41s)   kubelet            Pulling image "quay.io/jetstack/cert-manager-acmesolver:v1.1.0"
  Warning  Failed          6m53s (x3 over 7m40s)   kubelet            Failed to pull image "quay.io/jetstack/cert-manager-acmesolver:v1.1.0": rpc error: code = Unknown desc = no matching manifest for windows/amd64 10.0.17763 in the manifest list entries
  Warning  Failed          2m40s (x24 over 7m36s)  kubelet            Error: ImagePullBackOff

So I get no matching manifest for windows/amd64 becuase it was assigned to a windows node.
The same issue existed in the chart for ingress-nginx but there they have now set it default to linux becuase it can only run on linux. I would agree that it could be the same here too unless cert-manager is planning to support running on windows nodes.

While I agree this should be fixed by default, there is a way of setting this in the current release.
The spec.acme.solvers[].http01.ingress.podTemplate.spec.nodeSelector field can be set in the Issuer / ClusterIssuer object to add this nodeSelector. The helm chart isn't doing this because it doesn't create any issuers, that's left to the user. See https://cert-manager.io/docs/configuration/acme/http01/#podtemplate
Hopefully that provides a better way of working around this right now, than setting up taints etc.

While I agree this should be fixed by default, there is a way of setting this in the current release.
The spec.acme.solvers[].http01.ingress.podTemplate.spec.nodeSelector field can be set in the Issuer / ClusterIssuer object to add this nodeSelector. The helm chart isn't doing this because it doesn't create any issuers, that's left to the user. See https://cert-manager.io/docs/configuration/acme/http01/#podtemplate
Hopefully that provides a better way of working around this right now, than setting up taints etc.

Sorry I overlooked that config. I'll test it as quickly as possible.
Nevertheless, I think all of us agree that this should be set by default?

Sorry I overlooked that config. I'll test it as quickly as possible.
Nevertheless, I think all of us agree that this should be set by default?

Hi!
The nodeSelector field in the Helm chart seems to be made for the cert-manager's deployment itself (as opposed to the challenge pods which are created dynamically using the Issuer/ClusterIssuer configuration).
Might be worth adding this bit of information to the nodeSelector field in the helm chart's README.md, e.g.:
diff --git a/deploy/charts/cert-manager/README.template.md b/deploy/charts/cert-manager/README.template.md
index a6515327d..77c739af8 100644
--- a/deploy/charts/cert-manager/README.template.md
+++ b/deploy/charts/cert-manager/README.template.md
@@ -101,7 +101,7 @@ The following table lists the configurable parameters of the cert-manager chart
 | `securityContext` | Optional security context. The yaml block should adhere to the [SecurityContext spec](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.16/#securitycontext-v1-core) | `{}` |
 | `securityContext.enabled` | Deprecated (use `securityContext`) - Enable security context | `false` |
 | `containerSecurityContext` | Security context to be set on the controller component container | `{}` |
-| `nodeSelector` | Node labels for pod assignment | `{}` |
+| `nodeSelector` | Node labels for the assignment of the cert-manager controller. This field does not influence the assignment of ACME challenge pods; you can set the nodeSelector for the ACME challenge pods in the Issuer/ClusterIssuer object's `spec.acme.solvers[].http01.ingress.podTemplate.spec.nodeSelector` field. | `{}` |
 | `affinity` | Node affinity for pod assignment | `{}` |
 | `tolerations` | Node tolerations for pod assignment | `[]` |
 | `ingressShim.defaultIssuerName` | Optional default issuer to use for ingress resources |  |

What do you think?
/triage support

Is this solved?

/remove-lifecycle rotten
Seems still unresolved?

I just came across this issue and would agree with @maelvls' suggestion to add it to the documentation. Perhaps even the troubleshooting section of the site.

/remove-lifecycle stale
Still an issue


⚠ Bofrost.at has degraded performance

In 2cb7ce4, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 976 ms

Resolved: Bofrost.at performance has improved in de45c90 after 11 minutes.


Added Universidad de San Pedro Sula

@WMLondo Thank you for your patience while waiting for a review and processing your request. Pull request merged. Please start applying for JetBrains educational subscription at https://www.jetbrains.com/shop/eform/students in an hour, to allow the changes to propagate through JetBrains systems.


TypeScript source conversion

My first attempt to convert source to TypeScript per https://github.com/vkurchatkin/koa-connect/issues/11
I made the middleware function req and res types any because I wasn't really sure what they are expected to be. Initially I tried making them IncomingMessage and ServerResponse but this is not, for example, what connect-history-api-fallback uses.
I added scripts start and build to run the TS compiler (start in --watch). I'm not sure if build should be called in prepublish or something?

Thank you for this PR @aaronbeall, and extraordinary patience in waiting for a review.
Please take a look at the additions I made. Namely, I specified some types a little more strictly and converted the tests to TypeScript as well. I wouldn't feel good about converting the source to TypeScript without having the tests exercise the types.
Regarding the any types for req and res, I'm pretty sure you were on the right path with them being IncomingMessage and ServerResponse... unless Connect/Express use something else? I went ahead and used those, I'd rather not leave any any 🙃

@lourd Changes look good to me!

Just added an integration test to double-check that this would actually help in a practical scenario. I'm not sure it does... It requires casting ctx.req to any to read the body parameter; i.e. the added types from middlewares don't make it through to subsequent middlewares.
I'm still really unsure about adding these changes, as I have never personally used Koa or Express in a project written with TypeScript. It doesn't seem like this PR would improve the situation that much given the observed behavior. Someone please correct me or the code if that's not the case, but as it stands I don't think this should be merged.

Typescript requires types to be able to use the library at all. So the alternative is to add your own types which is what I am doing. Because I am lazy I just add it as any for everything. So for me I already have any everywhere for this module. So just having integrated types with any everywhere is a step forward for me because then I don't have to add that myself :-). I think the first step is just adding the types. Then it can of course be iterated on further in more PRs.

@lourd My two cents: unfortunately I'm no longer actively working on the project where I was using koa-connect, but I can say that the types I had written did indeed use any for req and res because otherwise it would raise type errors in my use case, and I couldn't really sort out what types those should be. This is an example where dynamic objects in JS make things tricky. It's like doable in TS, I just am not familiar enough with all the ways this should work in real use cases. However even with any for req and res it was still better than no types at all, which led to compiler type errors due to missing module definition and "implicit any" errors.

Alright, I agree. Better than no types at all 👍

Published in version 2.1.0


Jolt is empty? DB error root

Hey,
i wanted to check out the app but it is not working for me.
The login works fine, but the app and pages are empty.
I think there is a problem with the database. I used nearly complete the default config:
    04/15/2023 11:01:41 AM
    PostgreSQL Database directory appears to contain a database; Skipping initialization
    04/15/2023 11:01:41 AM
    04/15/2023 11:01:41 AM
    2023-04-15 09:01:41.682 UTC [1] LOG:  starting PostgreSQL 14.7 (Debian 14.7-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
    04/15/2023 11:01:41 AM
    2023-04-15 09:01:41.683 UTC [1] LOG:  listening on IPv4 address "<IP_ADDRESS>", port 5432
    04/15/2023 11:01:41 AM
    2023-04-15 09:01:41.683 UTC [1] LOG:  listening on IPv6 address "::", port 5432
    04/15/2023 11:01:41 AM
    2023-04-15 09:01:41.700 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
    04/15/2023 11:01:41 AM
    2023-04-15 09:01:41.773 UTC [27] LOG:  database system was shut down at 2023-04-10 19:35:22 UTC
    04/15/2023 11:01:42 AM
    2023-04-15 09:01:42.752 UTC [1] LOG:  database system is ready to accept connections
    04/15/2023 11:02:10 AM
    2023-04-15 09:02:10.860 UTC [41] FATAL:  role "root" does not exist
    04/15/2023 11:02:40 AM
    2023-04-15 09:02:40.990 UTC [51] FATAL:  role "root" does not exist
    04/15/2023 11:03:11 AM
    2023-04-15 09:03:11.132 UTC [60] FATAL:  role "root" does not exist
    04/15/2023 11:03:41 AM
    2023-04-15 09:03:41.279 UTC [69] FATAL:  role "root" does not exist
    04/15/2023 11:04:11 AM
    2023-04-15 09:04:11.404 UTC [79] FATAL:  role "root" does not exist

Hey. Can you provide some logs from Jolt? Currently the app doesn't show results on every page, but the latest release should yield results on the discover page.
Your feed will only show items, if someone else recommends media to you (or you use the playground to do it yourself).

You also might be running on an older version of Jolt. Try the develop tag if you want to verify that.

I did download the new release package or is there already a dockerhub repo?
`
...
04/15/2023 11:44:29 AM
caller=fxevent/zap.go:152level=infomsg=startedts=1681551869.5801315

04/15/2023 11:44:39 AM
[cors] 2023/04/15 09:44:39 Handler: Actual request
04/15/2023 11:44:39 AM
[cors] 2023/04/15 09:44:39   Actual request no headers added: missing origin
04/15/2023 11:44:39 AM
[cors] 2023/04/15 09:44:39 Handler: Actual request
04/15/2023 11:44:39 AM
[cors] 2023/04/15 09:44:39   Actual request no headers added: missing origin
04/15/2023 11:44:39 AM
[cors] 2023/04/15 09:44:39 Handler: Actual request
04/15/2023 11:44:39 AM
[cors] 2023/04/15 09:44:39   Actual response added headers: map[Access-Control-Allow-Credentials:[true] Access-Control-Allow-Origin:[*] Vary:[Origin]]
04/15/2023 11:44:39 AM
caller=graph/auth.resolvers.go:73elapsed=0.001102578error=record not foundlevel=errormsg=tracerows=0sql=SELECT * FROM "refresh_tokens" WHERE id = 4 AND "refresh_tokens"."deleted_at" IS NULL ORDER BY "refresh_tokens"."id" LIMIT 1stacktrace=github.com/dan6erbond/jolt-server/graph.(*mutationResolver).RefreshTokens /build/graph/auth.resolvers.go:73 github.com/dan6erbond/jolt-server/graph/generated.(*executionContext)._Mutation_refreshTokens.func2 /build/graph/generated/generated.go:2503 github.com/99designs/gqlgen/graphql/executor.processExtensions.func4 /go/pkg/mod/github.com/99designs/gqlgen@v0.17.20/graphql/executor/extensions.go:72 github.com/dan6erbond/jolt-server/graph/generated.(*executionContext)._fieldMiddleware /build/graph/generated/generated.go:1622 github.com/dan6erbond/jolt-server/graph/generated.(*executionContext)._Mutation_refreshTokens /build/graph/generated/generated.go:2501 github.com/dan6erbond/jolt-server/graph/generated.(*executionContext)._Mutation.func2 /build/graph/generated/generated.go:8651 github.com/99designs/gqlgen/graphql/executor.processExtensions.func3 /go/pkg/mod/github.com/99designs/gqlgen@v0.17.20/graphql/executor/extensions.go:69 github.com/dan6erbond/jolt-server/graph/generated.(*executionContext)._Mutation /build/graph/generated/generated.go:8650 github.com/dan6erbond/jolt-server/graph/generated.(*executableSchema).Exec.func2 /build/graph/generated/generated.go:998 github.com/99designs/gqlgen/graphql/executor.(*Executor).DispatchOperation.func1.1.1 /go/pkg/mod/github.com/99designs/gqlgen@v0.17.20/graphql/executor/executor.go:119 github.com/99designs/gqlgen/graphql/executor.processExtensions.func2 /go/pkg/mod/github.com/99designs/gqlgen@v0.17.20/graphql/executor/extensions.go:66 github.com/99designs/gqlgen/graphql/executor.(*Executor).DispatchOperation.func1.1 /go/pkg/mod/github.com/99designs/gqlgen@v0.17.20/graphql/executor/executor.go:118 github.com/99designs/gqlgen/graphql/handler/transport.POST.Do /go/pkg/mod/github.com/99designs/gqlgen@v0.17.20/graphql/handler/transport/http_post.go:55 github.com/99designs/gqlgen/graphql/handler.(*Server).ServeHTTP /go/pkg/mod/github.com/99designs/gqlgen@v0.17.20/graphql/handler/server.go:121 github.com/dan6erbond/jolt-server/pkg/auth.Middleware.func1.1 /build/pkg/auth/middleware.go:25 net/http.HandlerFunc.ServeHTTP /usr/local/go/src/net/http/server.go:2084 github.com/rs/cors.(*Cors).Handler.func1 /go/pkg/mod/github.com/rs/cors@v1.8.2/cors.go:231 net/http.HandlerFunc.ServeHTTP /usr/local/go/src/net/http/server.go:2084 github.com/gorilla/mux.(*Router).ServeHTTP /go/pkg/mod/github.com/gorilla/mux@v1.8.0/mux.go:210 net/http.serverHandler.ServeHTTP /usr/local/go/src/net/http/server.go:2916 net/http.(*conn).serve /usr/local/go/src/net/http/server.go:1966ts=1681551879.873334

add caller=graph/auth.resolvers.go:73
add elapsed=0.001102578
add error=record not found
add level=error
add msg=trace
add rows=0
add sql=SELECT * FROM "refresh_tokens" WHERE id = 4 AND "refresh_tokens"."deleted_at" IS NULL ORDER BY "refresh_tokens"."id" LIMIT 1
add stacktrace=github.com/dan6erbond/jolt-server/graph.(*mutationResolver).RefreshTokens /build/graph/auth.resolvers.go:73 github.com/dan6erbond/jolt-server/graph/generated.(*executionContext)._Mutation_refreshTokens.func2 /build/graph/generated/generated.go:2503 github.com/99designs/gqlgen/graphql/executor.processExtensions.func4 /go/pkg/mod/github.com/99designs/gqlgen@v0.17.20/graphql/executor/extensions.go:72 github.com/dan6erbond/jolt-server/graph/generated.(*executionContext)._fieldMiddleware /build/graph/generated/generated.go:1622 github.com/dan6erbond/jolt-server/graph/generated.(*executionContext)._Mutation_refreshTokens /build/graph/generated/generated.go:2501 github.com/dan6erbond/jolt-server/graph/generated.(*executionContext)._Mutation.func2 /build/graph/generated/generated.go:8651 github.com/99designs/gqlgen/graphql/executor.processExtensions.func3 /go/pkg/mod/github.com/99designs/gqlgen@v0.17.20/graphql/executor/extensions.go:69 github.com/dan6erbond/jolt-server/graph/generated.(*executionContext)._Mutation /build/graph/generated/generated.go:8650 github.com/dan6erbond/jolt-server/graph/generated.(*executableSchema).Exec.func2 /build/graph/generated/generated.go:998 github.com/99designs/gqlgen/graphql/executor.(*Executor).DispatchOperation.func1.1.1 /go/pkg/mod/github.com/99designs/gqlgen@v0.17.20/graphql/executor/executor.go:119 github.com/99designs/gqlgen/graphql/executor.processExtensions.func2 /go/pkg/mod/github.com/99designs/gqlgen@v0.17.20/graphql/executor/extensions.go:66 github.com/99designs/gqlgen/graphql/executor.(*Executor).DispatchOperation.func1.1 /go/pkg/mod/github.com/99designs/gqlgen@v0.17.20/graphql/executor/executor.go:118 github.com/99designs/gqlgen/graphql/handler/transport.POST.Do /go/pkg/mod/github.com/99designs/gqlgen@v0.17.20/graphql/handler/transport/http_post.go:55 github.com/99designs/gqlgen/graphql/handler.(*Server).ServeHTTP /go/pkg/mod/github.com/99designs/gqlgen@v0.17.20/graphql/handler/server.go:121 github.com/dan6erbond/jolt-server/pkg/auth.Middleware.func1.1 /build/pkg/auth/middleware.go:25 net/http.HandlerFunc.ServeHTTP /usr/local/go/src/net/http/server.go:2084 github.com/rs/cors.(*Cors).Handler.func1 /go/pkg/mod/github.com/rs/cors@v1.8.2/cors.go:231 net/http.HandlerFunc.ServeHTTP /usr/local/go/src/net/http/server.go:2084 github.com/gorilla/mux.(*Router).ServeHTTP /go/pkg/mod/github.com/gorilla/mux@v1.8.0/mux.go:210 net/http.serverHandler.ServeHTTP /usr/local/go/src/net/http/server.go:2916 net/http.(*conn).serve /usr/local/go/src/net/http/server.go:1966
add ts=1681551879.873334`

Ah, thanks for the logs! That looks like a weird refresh tokens error. Since I just now published v0.5.2 could you flush your DB (just delete the Postgres volume) and redeploy? That should fix it!
The FATAL:  role "root" does not exist error happens because of the healthcheck in my Docker compose, I have to fix that as well. 😅

Still same error. Do i need to download the new release package?
About the TVDB Token, do i need to use the API Key v3 or the Read Access Token v4?

@Cebrain which version of Jolt are you deploying? The latest is v0.5.2.

I downloaded the latest package 0.5.2 and rebuild
jolt-app-1-2023-04-15T10-19-33.log.gz

Ohhh. I just realized the error probably has to do with a flushed DB.
I'll fix the error soon. In the meantime can you try clearing your localStorage/site data?

I deleted the ./.postgres folder, this should be enough?

I meant your browser data. You can click on the lock icon of the page URL, then remove all the site data.


[5.7] Remove unnecessary conditional

$this->getAlias($abstract) will return $abstract if it is not an alias.

No plans to change this unless there is a bug.


bad results, how to solve?

https://github.com/user-attachments/assets/a1edd7c9-5015-4070-8de7-2ffed9860908
The command is:
CUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --nnodes=1 --nproc_per_node 1 --master_port 29514
-m opensora.sample.sample
--model_type "inpaint"
--model_path "/home//llm/models/Open-Sora-Plan-v1.3.0/any93x640x640_i2v"
--version v1_3
--num_frames  33
--height 352
--width 640
--max_hxw 236544
--crop_for_hw
--cache_dir "../cache_dir"
--text_encoder_name_1 "/home//llm/models/mt5-xxl"
--text_prompt examples/cond_prompt.txt
--conditional_pixel_values_path examples/cond_pix_path.txt
--ae WFVAEModel_D8_4x8x8
--ae_path "/home/*******/llm/models/Open-Sora-Plan-v1.3.0/vae"
--save_img_path "./save_path"
--fps 6
--guidance_scale 7.5
--num_sampling_steps 100
--max_sequence_length 512
--sample_method EulerAncestralDiscrete
--seed 1234
--num_samples_per_prompt 1
--rescale_betas_zero_snr
--prediction_type "v_prediction"
--noise_strength 0.0 \

Due to the data used during training, selecting num_frames=93 will yield relatively better results. Additionally, because of the current model's limited capability, inference in multi-person scenarios has a higher failure rate. This can be mitigated by adjusting the seed and prompt multiple times to achieve the best results. We are working on training a new version of the Open-Sora Plan model to provide better generation quality. Thank you for your understanding.

how to set --text_prompt examples/cond_prompt.txt?


Handle do while statements

Fixes https://github.com/thorbenvh8/prettier-java/issues/56

sorry that the master is broken right now!
But your code looks fine! Thanks for the work!


GO:0034050 missing parent

GO:0034050    JSON
host programmed cell death induced by symbiont
has no parentage to
GO:0051702   interaction with symbiont

Looking at the annotations, I put this under 'defense response to other organism' (we dont have 'defense response to symbiont', so I also added GO:0051702 (now called 'biological process involved in interaction with symbiont').
Should I create 'defense response to symbiont'?
Thanks, Pascale

Also -> changed label from 'host programmed cell death induced by symbiont' to 'programmed cell death induced by symbiont'
but I wonder if it would not be more clear to have 'programmed cell death in defense to symbiont' or something along those lines ?
Thanks, Pascale

yes,
'programmed cell death in defense to symbiont' would be much clearer.
'defense response to symbiont'? is not a bad idea.
It would probably be needed if  we eventually get rid of the organism specific ones in favour of some better method


Update the contributing guide.

@Vyvy-vi Here, I rebased my branch on your main branch, and pushed my changes again.

@all-contributors add @anubhavcodes docs


vim-mode-plus support

I'm not sure how/whether to test this. It seems to be compatible so far.

Having just run the tests against vim-mode-plus, this is more incompatible than I'd hoped. It's probably easiest for me to fork.


Remove insignificant zeros in Faker::Number

Right now if you do Faker::Number.number(5) then you can get a string like "01234" which when turned into a number will end up as 1234 - ie. not a 5 digit number at all.  Likewise Faker::Number.decimal(5) can get a string like "12345.60" which will turn into 12345.6 which is also not generally what people will expect.
This PR fixes that for all argument values above 1, when the arg is 1 then I allow a single "0" to be returned, because I figure people who want a single digit will probably want the possibility of a zero.
Tests included too.

Thanks.