Unexpected result from a join to a lookup table

I have two models; Packs and Locales. A Pack has a localeId which is a FK to Locales. Locales is just a lookup table. I've defined the models like
  let Pack = bookshelf.Model.extend({
    tableName: 'packs',
    hasTimestamps: ['createdAt', 'updatedAt'],
    locale() {
      return this.belongsTo('Locale', 'id');
    }
  });

  let Locale = bookshelf.Model.extend({
    tableName: 'locales',
    hasTimestamps: ['createdAt', 'updatedAt']
  });

I have two Packs, both with a localeId of 1 and the result I get from a fetchAll is:
[{ id: 1,
    name: 'United Kingdom',
    archived: false,
    createdAt: Mon Jan 18 2016 09:12:45 GMT+0000 (GMT),
    updatedAt: Mon Jan 18 2016 09:12:45 GMT+0000 (GMT),
    localeId: 1,
    locale:
     { id: 1,
       code: 'en-GB',
       archived: false,
       createdAt: Mon Jan 18 2016 09:12:45 GMT+0000 (GMT),
       updatedAt: Mon Jan 18 2016 09:12:45 GMT+0000 (GMT) }
{ id: 13,
    name: 'United Kingdom',
    archived: false,
    createdAt: Tue Jan 26 2016 13:30:15 GMT+0000 (GMT),
    updatedAt: Tue Jan 26 2016 13:30:15 GMT+0000 (GMT),
    localeId: 1,
    locale: {},
 }]

I'd expect both Pack objects to have a Locale object, but only the first does. Is this something I've done incorrectly in my model definitions? Or a bug?

That seems like a bug, but can you provide the code that leads to that result?

No probs, just a simple fetchAll is enough to replicate it:
this.Model // where this.Model === the Pack model
      .fetchAll({
        withRelated: ['locale']
      });

That returns the result I posted in the OP.

This is a user error:
    locale() {
      return this.belongsTo('Locale', 'id');
    }

Should be:
    locale() {
      return this.belongsTo('Locale', 'localeId');
    }

You're supplying the wrong columns for the join. Hence it's joining locales.id to packs.id. By coincidence these have the same value of 1 in the first model of your example. Make sure to test with debug: true option to get debug output.


Uncaught TypeError: Invalid Point: (163, NaN)

[Enter steps to reproduce:]

... OPENED A FILE FROM MY NETWORK
... JUST PRESSED [BACKSPACE ON A SPECIFIC LINE THAT I WANTED TO REMOVE FROM THE FILE]

Then this Red Modal appeard
Atom: 1.26.1 x64
Electron: 1.7.11
OS: Microsoft Windows 10 Home
Thrown From: Atom Core
Stack Trace
Uncaught TypeError: Invalid Point: (163, NaN)
At C:\Users\space_000\AppData\Local\atom\app-1.26.1\resources\app\node_modules\text-buffer\lib\text-buffer.js:959

TypeError: Invalid Point: (163, NaN)
    at Function.module.exports.Point.assertValid (~/AppData/Local/atom/app-1.26.1/resources/app/node_modules/text-buffer/lib/point.js:63:21)
    at DisplayLayer.translateScreenPosition (~/AppData/Local/atom/app-1.26.1/resources/app/node_modules/text-buffer/lib/display-layer.js:345:17)
    at DisplayMarkerLayer.module.exports.DisplayMarkerLayer.translateScreenPosition (~/AppData/Local/atom/app-1.26.1/resources/app/node_modules/text-buffer/lib/display-marker-layer.js:163:38)
    at DisplayMarker.module.exports.DisplayMarker.setHeadScreenPosition (~/AppData/Local/atom/app-1.26.1/resources/app/node_modules/text-buffer/lib/display-marker.js:154:58)
    at changePosition (~/AppData/Local/atom/app-1.26.1/resources/app/src/cursor.js:68:25)
    at Cursor.changePosition (~/AppData/Local/atom/app-1.26.1/resources/app/src/cursor.js:713:11)
    at Cursor.setScreenPosition (~/AppData/Local/atom/app-1.26.1/resources/app/src/cursor.js:67:16)
    at Cursor.moveLeft (~/AppData/Local/atom/app-1.26.1/resources/app/src/cursor.js:288:18)
    at modifySelection (~/AppData/Local/atom/app-1.26.1/resources/app/src/selection.js:252:50)
    at Selection.modifySelection (~/AppData/Local/atom/app-1.26.1/resources/app/src/selection.js:984:11)
    at Selection.selectLeft (~/AppData/Local/atom/app-1.26.1/resources/app/src/selection.js:252:16)
    at Selection.backspace (~/AppData/Local/atom/app-1.26.1/resources/app/src/selection.js:496:36)
    at mutateSelectedText.selection (~/AppData/Local/atom/app-1.26.1/resources/app/src/text-editor.js:1373:65)
    at getSelectionsOrderedByBufferPosition.map (~/AppData/Local/atom/app-1.26.1/resources/app/src/text-editor.js:1387:92)
    at Array.map (native)
    at transact (~/AppData/Local/atom/app-1.26.1/resources/app/src/text-editor.js:1387:66)
    at TextBuffer.module.exports.TextBuffer.transact (~/AppData/Local/atom/app-1.26.1/resources/app/node_modules/text-buffer/lib/text-buffer.js:952:24)
    at TextEditor.transact (~/AppData/Local/atom/app-1.26.1/resources/app/src/text-editor.js:1839:30)
    at mergeIntersectingSelections (~/AppData/Local/atom/app-1.26.1/resources/app/src/text-editor.js:1386:25)
    at TextEditor.mergeSelections (~/AppData/Local/atom/app-1.26.1/resources/app/src/text-editor.js:3252:26)
    at TextEditor.mergeIntersectingSelections (~/AppData/Local/atom/app-1.26.1/resources/app/src/text-editor.js:3223:42)
    at TextEditor.mutateSelectedText (~/AppData/Local/atom/app-1.26.1/resources/app/src/text-editor.js:1385:23)
    at TextEditor.backspace (~/AppData/Local/atom/app-1.26.1/resources/app/src/text-editor.js:1373:23)
    at TextEditor.object.(anonymous function) [as backspace] (~/AppData/Local/atom/app-1.26.1/resources/app/node_modules/underscore-plus/lib/underscore-plus.js:77:33)
    at TextEditor.core:backspace (~/AppData/Local/atom/app-1.26.1/resources/app/src/register-default-commands.js:442:27)
    at ~/AppData/Local/atom/app-1.26.1/resources/app/src/register-default-commands.js:701:40
    at TextBuffer.module.exports.TextBuffer.transact (~/AppData/Local/atom/app-1.26.1/resources/app/node_modules/text-buffer/lib/text-buffer.js:952:24)
    at TextEditor.transact (~/AppData/Local/atom/app-1.26.1/resources/app/src/text-editor.js:1839:30)
    at HTMLElement.newCommandListeners.(anonymous function) (~/AppData/Local/atom/app-1.26.1/resources/app/src/register-default-commands.js:700:28)
    at CommandRegistry.handleCommandEvent (~/AppData/Local/atom/app-1.26.1/resources/app/src/command-registry.js:384:49)

Commands
     -3:18.7.0 tabs:close-all-tabs (div.title)
     -0:30.5.0 core:backspace (input.hidden-input)
 21x -0:28.1.0 core:select-left (input.hidden-input)
     -0:26 core:backspace (input.hidden-input)
     -0:23.8.0 editor:select-to-first-character-of-line (input.hidden-input)
  2x -0:23.4.0 core:backspace (input.hidden-input)

Non-Core Packages
atom-music 0.7.0
atom-play-piano 3.2.1
color-picker 2.3.0
emmet 2.4.3
highlight-selected 0.13.1
html-entitize 1.1.0
language-asp-html 0.3.0
language-vb 0.1.0-pre1

Thanks for contributing!
We noticed that this looks like a duplicate of https://github.com/atom/atom/issues/14774 so you can subscribe there for updates if you'd like.  If you can consistently reproduce the error, please let us know in https://github.com/atom/atom/issues/14774.

Because we treat our issues list as the Atom team's backlog, we close duplicates to focus our work and not have to touch the same chunk of code for the same reason multiple times. This is also why we may mark something as duplicate that isn't an exact duplicate but is closely related.
For information on how to use GitHub's search feature to find out if something is a duplicate before filing, see the How Can I Contribute? section of the Atom CONTRIBUTING guide.


Add ForbiddenNamedParam rule

Fixes https://github.com/detekt/detekt/issues/7349

Messages

:book:
Thanks for adding a new rule to detekt :heart:

:white_check_mark:
We detekted that you added tests, to your rule, that's awesome!

:white_check_mark:

We detekted that you updated the default-detekt-config.yml file, that's awesome!

  Generated by :no_entry_sign: dangerJS against efa994ac85758c9c40eea46853f2144cae934651

Nice one! Thanks for the contribution.


(maint) Add tests for roundtripping hash serialisation

Previously the Sidecar Protocol had methods to convert to and from hash however
they were never tested to ensure that they could be "round-tripped", that is
converted to a hash and then back to a sidecar object with no loss of data. This
commit adds the test and corrects any behvaiour in the protocol implementation.

Self merging


Question about transaction timestamp

Does transaction execution or confirmation timestamp exists in "Libra" blockchain? For  example on Ethereum there is a timestamp when the block was mined. Does "Libra" transaction contains something similar and is it possible to retrieve it using "Libra client". The transaction "expiration date" I assume has a different meaning.

There is a timestamp created when a block is proposed.  The block timestamp is guaranteed to be higher than the preceding parent block's timestamp.  It is also guaranteed that if the block is accepted, that > 2/3 of the voting power agreed that timestamp was in the past.  We will be working to expose the timestamps to clients prior to V1 launch and on other improvements to the timestamp protocol.  Transaction expiration is related in the sense that transactions won't be proposed (or accepted) if they violate the timestamp protocol - i.e. older than the block's timestamp.

So how do you get timestamp of a transaction? I am assuming (based on response) tx timestamp is currently not available and if so a fairly significant limitation.


Fix scrollToTop again

📲 What
Use optionals to prevent from possible crash caused by accessing implicitly unwrapped optional.
🤔 Why
We're still seeing couple crashes related to the scrollToTop issue. Due to the fact that our crash reporting tools indicated the problem was only related to BackerDashboardViewController it was the only controller fixed at the time. Since the same implementation is also used in DiscoveryViewController it's only adequate to also address it here.
🛠 How
Using optionals when extracting a reference to a view that might be on a controller that might not be allocated due to being declared as implicitly unwrapped optional.
👀 See
https://github.com/kickstarter/ios-oss/pull/516
✅ Acceptance criteria

[ ] Explore screen still scrolls to top when tapping the status bar
[ ] Same behaviour works for different filter (Projects We Love, Popular, Newest, etc.)

 Scroll to top crash

@dusi from what I recall, our previous hypothesis on this crash had to do with a potential memory leak on VCs storing weak references to pageViewControllers. I re-opened the trello card with a reference to the crash hoping we could take a closer look at the leak in a future sprint. This temporary solution looks good for now, but maybe we should make sure we address the root cause in an upcoming sprint.

Thanks for looking into this @dusi.
Just to summarize from our discussion on Slack:

The likely cause of this appears to be that some of our VC's are leaking.
We should try to reproduce using the breadcrumb logs that we have in the crash reports, they all appear to crash when the user selects the currency cell after logging out and in again (and doing a bunch of stuff in between).
This seems related to us previously propagating the ksr_userLocalePreferencesChanged notification when the user updates their currency. This was removed in #575 but should be considered a regression and be brought back.
If we can reproduce it this way, we should try to prove that it is or isn't caused by the stuff that we're doing in RootViewModel with signals emitting our root view controllers: https://github.com/kickstarter/ios-oss/blob/master/Kickstarter-iOS/ViewModels/RootViewModel.swift#L108. I believe that this could be problematic and we should look at ways to change this so that signals are not retaining view controllers. Perhaps instead we should emit enum cases representing the view controllers to create rather than the view controllers themselves.


Set up RSpec integration with VSCode

I'm using RSpec, but for ease of use and debugging it would be good to integrate it into VSCode's Docker-based dev-container environment as I'm using that for development. When this is done it should be possible to:

[x] Run all specs from the VSCode UI, and display results
[x] Run a specific spec file from the UI, and display results
[x] Run a select spec from a file, from the UI, and display results
[x] Ensure that db changes aren't persisted following spec completion

Integrating automated spec-running with the GitHub repo is a later exercise.

All done by installing an extra couple of extensions. No need to change the code in any way. I've added the test extensions to the default dev-container setup anyway, though.


Main

Feature- Interactive cybersecurity quiz
The quiz covers various important topics through multiple-choice questions. Users get instant feedback on their answers, helping them learn as they go, and can track their progress over time. A leaderboard is included to encourage some friendly competition, and after completing the quiz, users can explore helpful resources also. The quiz is designed to be easy to use, responsive, and engaging for all users.
For better understanding of the code, I have added a separate folder named as interactive_quiz consisting of files index.html, styles.css, quiz.js
I request you to to kindly merge my code. I will be grateful to you.

Hi Thanks for the PR
If you want, you can mention the issue number you're addressing in your PR using one of several keywords and GitHub will automatically link them together for you.
https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue

Hey @B1N4RY-P4R45173 you asked me about my issue number and all but you had not merged my pr yet..Is there any specific reason for the same?

@Vinay-Khanagavi  Can you please check and review this?
Also @snehas-05 you have to update the repo and resolve conflicts for us to be able to merge your pr

Is there any issue?

I don't see any issue but the right person to comment on this is @Vinay-Khanagavi. Since he is looking after these kinds of issues
As of now you just need to sync your repo with the current commits without which we cannot merge. You can refer to
https://www.atlassian.com/git/tutorials/merging-vs-rebasing if you don't know how to do it. And for the review pls wait till Monday. If @Vinay-Khanagavi doesn't respond by then I will review it. Sorry for this delay.

Okay no issues. I will sync the changes after going through the steps once from the link you provided.

Hey @B1N4RY-P4R45173 I have synced my repo can you please check once?


Support dictionary/data lookups of text

This adds support for looking up data under the mouse cursor. Usually it will bring up a dictionary, but other times it could be a Wikipedia article, Siri knowledge, etc. Apple doesn't really have a good name for it, other than "looking up data", "quick look" (a confusingly similar name with the other Quick Look OS feature), or "show definition". You can activate this by doing Ctrl-Cmd-D when the mouse is over a cursor. If you have a trackpad, you can also either activate this using Force click or three-finger tap (depends on your system preference settings).
Note that for Force click, this could potentially make it impossible to use the MacVim <ForceClick> mapping in Vim, which allows you to map a force click to a Vim command (#716). This is handled by having a new setting (under a new "Input" preference pane which will have more populated later) that allows you to choose whether to use Force click for data lookup or Vim's <ForceClick> mapping. If you have configured to use three-finger taps though this setting wouldn't do anything, and <ForceClick> is always send to the Vim mapping.
Also, this is lacking a lot of features that a normal macOS application would get, e.g. looking up selected texts (e.g. if you have "ice cream", you may want to select the whole thing to look up the phrase, rather than just "ice" or "cream"), data detector, and much more (e.g. custom API support). They will be done later as part of #1311.
Technical details below
The way the OS knows how to look up the data and present it is by talking to the NSTextInput/NSTextInputClient. Previously MacVim implemented NSTextInput partially, and didn't implement the critical firstRectForCharacterRange:actualRange and characterIndexForPoint: functions. First, in this change we change from NSTextInput to NSTextInputClient (which is the newer non-deprecated version), and implement those functions, which allows the OS to query the text storage.
By default, the OS sends a quickLookWithEvent: call to us whenever the lookup happens but for some odd reason this isn't automatic for Force clicks, presumably because some apps want to handle Force clicks manually (this is why some apps only work for three-finger taps but not Force clicks for lookups). This isn't documented but I found references in iTerm/Firefox, and basically we just need to manually handle it and send off quickLookWithEvent: when handling Force clicks.
For implementing the NSTextInputClient properly, the main issue is making sure that can work properly with input methods / marked texts, which is the other primary purpose for this class (other than inputting keys). For data lookups, I'm representing the grid as a row-major text (with no newline/space in between) and expose that to the OS. This already has some issue because it doesn't handle Vim vertical splits well, as MacVim doesn't really have access to detailed Vim text buffers easily (unless we do a lot of calls back-and-forth). This means wrapped texts won't be looked up properly, which I think is ok. Also, the OS APIs deal with UTF-8 indices, so we can't just convert row/column to raw indices and have to do a lot of character length calculations (especially for wide chars like CJK or emojis) to make sure the returned ranges are consistent and valid. For marked texts though, this presents a challenge because Vim doesn't really have a strong enough API to communicate back-and-forth about the marked positions and whatnot (it only let the GUI know where the current cursor is), and it's hard to implement APIs like markedRange properly because some marked texts could be hidden or wrapped (if you implement some of these functions improperly Apple's input methods could start misbehaving especially when you use arrow keys to navigate). In the end I kept the original implementation for treating the marked texts as a range starting from 0, only when we have marked text. Kind of a hack but this makes sure we work both in marked text mode (i.e. when inputting texts) and when doing lookups. For simplicity I made it so that you can't do data lookups when in marked text mode now.
Input method
This change also fixes a quirk in input method as a driveby change. Previously the logic for calculating the rect for where the candidate list was quite broken, but now it's calculated correctly using the desired range and the current cursor position. This matters when say using Japanese IM and using the left/right arrow to jump to different sections of the text. If the desired range is in a wrapped line, the new logic would attempt to pin it to the left-most column of where the cursor is in the range.
Data detection
Note that the default implementation is quite bare, and lacks a lot of smart data detection. For example, if you put your mouse over a URL, it won't properly select the whole URL (since the word delineation usually stops at the ":"), and addresses and dates for example also won't get grouped together properly. This is because these require additional implementation (e.g. using NSDataDetector) instead of coming "for free", and will be handled later. In fact, Apple's WebKit and NSTextView cheats by calling an internal API framework called "Reveal" (which you can find out by intercepting NSTextView's calls and/or looking at WebKit's source code) which is much more powerful and supports looking up package tracking, airline info, and more, but it's not available to third-party software (that's why Safari's lookup is so much better than Chrome/Firefox's).

This isn't tested right now. Future task needs to add XCTest support to properly test this as there are a lot of edge cases involved here.

Fix #1191
Part of Epic #1311, which contains other items to be implemented.

Screenshot:

Input method fix where the candidate box is where it should be now when going left/right.


OSX installation docs are confusing as to which method to use

What version of Delve are you using (dlv version)?

$ dlv version
Delve Debugger
Version: 1.0.0
Build: $Id: c98a142125d0b17bb11ec0513bde346229b5f533 $

What version of Go are you using? (go version)?

go version go1.9.2 darwin/amd64

What operating system and processor architecture are you using?

OSX

What did you do?

Read https://github.com/derekparker/delve/blob/master/Documentation/installation/osx/install.md

What did you expect to see?

A clear explanation of what this means:

With this method you will not be able to use delve's native backend.

What did you see instead?

I think the docs would be improved if it explained the advantage of using delve's native backend.  Also, it might be better to explicitly reference which method, like "With the go get method .. ", since it's hard to tell if it's talking about the preceding or the following method.
edrozenberg was confused by it too, so it's not just me.

You can technically simplify the instructions into the below steps. Using -d the go get command will stop before it installs the binary in the bin directory, and you don't have to go out of your way to clone the repo into the right place for make install. While the make install step isn't strictly necessary, it does populate the Build value in delve version

Clone delve go get -u -d github.com/derekparker/delve/cmd/dlv
move to directory cd $GOPATH/src/github.com/derekparker/delve
install delve make install

I think we clarified this sufficiently in 910f90.


jltest: Add support for running tests in verbose mode.

The @testset macro makes it hard pass options defined at run-time (e.g., it's hard to pass verbose=true to @testset macro).
One possibility would be to use a global variable like jltest_verbose that gets passed as a keyword argument to to the constructor of the DefaultTestSet embedded within EnhancedTestSet.
EnhancedTestSet{T}(desc) where {T} = new(T(desc, verbose=jltest_verbose))

Using expression quoting is a better approach for setting the verbose option to the @testset macro.
https://docs.julialang.org/en/v1/manual/metaprogramming/

Basic implementation complete.
Next Steps

Add unit tests

Basic unit tests implemented.


Card secondary variant

Add secondary variant to Card (part of #2595)
Tasks
### Checklist
- [ ] Density (HD, MD, LD, TD)
- [ ] Mode (Light, Dark)
- [ ] Fill (Primary, Secondary, Tertiary)
- [ ] State (Default, Hover, Selected/Active, Disabled, Read-only, Other)
- [ ] Controlled API
- [ ] UITK accessibility parity (screen reader + keyboard interactions)

### Deliverables
- [ ] Characteristics spec
- [ ] Interactions spec, where necessary
- [ ] In Code & Figma (Component + Guide)
- [ ] Supporting site documentation

13 Nov - made some changes to PR comments, needs review


Docs: fix typo

Found a typo in sagas.js:

all is not a function but call is
parentheses of the saga-functions in the array are removed

What version of the redux-saga do u use?

0.14.8
LG
Bianca

Am 05.05.2017 um 16:17 schrieb Mateusz Burzyński<EMAIL_ADDRESS>What version of the redux-saga do u use?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/redux-saga/redux-saga/pull/968#issuecomment-299476754, or mute the thread https://github.com/notifications/unsubscribe-auth/AQ5C-a8wdq8qumWzNYmHe2mRIwnCVNj4ks5r2y95gaJpZM4NR7Do.

all was introduced in v0.15.0.
It's basically the same thing as 'parallel' effect - yield [eff, eff, eff]. However all is preferred way now to handle this as its more explicit and corresponds in naming to the Promise.all nicely.


Update coverage generation

Coverage increased (+8.8%) to 91.234% when pulling 19ef6469e4d8a866c428f543ee6e44ede8812af5 on update-coverage into 6402a47f9d8a02ee0069df716022acdab867e650 on master.


Need to add popup on error

Need to add popup on error

http://stackoverflow.com/questions/16651227/enable-angular-ui-tooltip-on-custom-events


Incomplete AST

I am trying to analyze a parsed document using Document?.Descendants() but some blocks like MathBlock even if they are parsed and rendered properly are not found in the AST.
I did a similar test with FencedCodeBlock and it works as expected.
What is the reason do not have them in the AST?

What is the reason do not have them in the AST?

Can you show a small repro with the exact setup of the pipeline you are using?


Update index.js

Prevent to redefine serializer if it was specified at function call.

The ternary should still work. Does it not? can you make a failing test?

Here it is. I've included your lib without fixes.

@proxyfabio sorry, I meant can you write a test in the test suite so we don't break this in the future :)

https://github.com/defunctzombie/form-serialize/pull/15


Provide defaults for material's normalScale and emissive values.

Provide for defaults for these properties when parsing them in updateFrom.
Ideally this behaves more like it did at https://github.com/gkjohnson/three-gpu-pathtracer/blob/4fc62182d4fe03c834515de388d0af7a6fb8ea24/src/uniforms/MaterialStructUniform.js

Great - thank you!


AVRO-3896: support to custom more logical type

What is the purpose of the change
For now, our process for logical types is:

parse the schema
check whether the schema is supported internally:

if yes, returun the internal schema (e.g. Decimal)
if no, return a error(this what logical_verify_type do) or return the schema directly(this what try_logical_type do)

Returning a error prevents user to custom the logical type, so this PR:

unify the process of all logical type: return the schema directly if the schema isn't supported internally.
Refine the process logical. (Hopefully it looks more clear. I'm glad to revert it if it doesn't look clear

Verifying this change
(Please pick one of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Extended interop tests to verify consistent valid schema names between SDKs
Added test that validates that Java throws an AvroRuntimeException on invalid binary data
Manually verified the change by building the website and checking the new redirect

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

2. Hopefully it looks more clear. I'm glad to revert it if it doesn't look clear

I tried to review this PR several times and all the time I got easily lost in the changes :-/
And without new unit tests it is hard to understand the use case and the desired behavior.

Hopefully it looks more clear. I'm glad to revert it if it doesn't look clear

I tried to review this PR several times and all the time I got easily lost in the changes :-/
And without new unit tests it is hard to understand the use case and the desired behavior.

Thanks for the review! Sorry, I add the unit test and rename the function to try to make it more clear. Please let me know if it's still confusing.

Thank you, @ZENOTME !


Display "More listings by..." section at the bottom of the listing detail screen

It would be nice to display additional listings sold by the vendor at the bottom of the listing detail screen. This is a good opportunity to help people discover more and keep them engaged in the shopping process.
Not urgent, but if it's relatively low effort, it would be worth adding.

This is pretty important feature to add as it helps users continue browsing easily without having to constantly hitting back. This issue has been open for 1.5y now without any progress. Any chance we can work it into the schedule?


Firestore Snapshot listener not receiving document changes

I'm trying to listen changes from a collection. The code below loads the initial data, but never gets triggered when a document is updated. I already have another code without the arrayContains condition and it's working well. Could it be arrayContains, which interferes with stream updates?
Reproduce
 final snapshots = _firestore.collection("posts")
          .where("category", arrayContains: 'technology')
          .orderBy("date", descending: true)
          .endBefore([date])
          .limit(10)
          .snapshots().

Here is the solution for people who have the same problem. It's about Indexes.

Open the database of your Firebase Console.
Go to the Indexes tab.
Click Add Index.
Add fields that you've been trying to listen.

It takes some time to create. After it is ready, you will be able to listen updates.
In addition to that, you don't have to create an index manually. You can get the direct link from the error message, which creates your index automatically. However, here is the trick. When I used snapshots(), I did not see the error message, but I was able to see it whenever I switch it to getDocuments(). So, use getDocuments() temporarily to get the direct link. You can change your code later. Hope that helps.

Thanks!

There is no error / index url displaying in console of Vs-code editor. Kindly suggest what should I do? BTW how can we delete already created indexes on Firestore?
BTW I have the same issue .limit(1).snapshots().listen((event) {... not working and not calling listen after adding new document / deleting existing document. Kindly share your suggestion. Thank you.

You can't get errors without setting an error handler when you use snapshots() method. See the following code to receive an error that contains a link to generate the required indexes:
FirebaseFirestore.instance
  .collection('users')
  .orderBy('age')
  .where("age", isGreaterThan: 30)
  .limit(10)
  .snapshots()
  .listen(
  (event) {
    print(event);
  },
  onError: (error) {
    print(error);
  },
);

How to remove indexes
Follow the instructions to remove an index:

Go to the Cloud Firestore section of the Firebase console.
Click the Indexes tab.
Hover over the index you want to delete and select Delete from the context menu.
Confirm that you want to delete it by clicking Delete from the alert.

Please see the documentation for details.


TSC Meeting #34 2020-11-18 20:00 UTC

Agenda

add topics here
Review TSC issue backlog
Review TSC-tagged issues (toggle to "open" after checking for recently "closed" items)

@ampproject/tsc
@tobie Can you specify AC attendees?

I've rolled forward the agenda from the Nov 18 meeting to our Dec 2 meeting because we did not meet due to lack of agenda.

Notes:

Malte suggested that we do a 2021 roadmap sync at next meeting

Rudy will follow up with @nainar

Review TSC issue backlog

https://github.com/ampproject/meta-tsc/issues/30

Joey will work on finding an owner
Ask AC to submit suggestions
Current proposal listed in the issue is for the TSC to own this
We’ll collect suggestions in a public GH issue
TSC’s next meeting will be in the new year. We’ll look at proposals then.

Review TSC-tagged issues (toggle to "open" after checking for recently "closed" items)


Support for MASM Comment Directive

Example:
comment ! [this line is commented]
	[multiline comments]
	[any symbol can be used as a delimiter, first instance of this symbol marks the end of the comment directive]
! [this line is also commented]

MSDN documentation: https://msdn.microsoft.com/en-us/library/4c9h3x38.aspx

I always wondered why it was necessary to have such a strong method in which any char could be redefined. It makes it hard to make a clean parser/tagger, let alone an efficient one. I guess that in the future I'll make a separate parser for Masm and another for Nasm. Once I've done that I will consider adding support for the "comment" keyword. I've added it to the TODO list.
For the moment, '#' and ';' give line comments.


Java Exception in Docker

I am trying to run UFO using docker but I keep getting exception when trying to run java -jar ufo-predict/runufo.jar
The error message is exception in "main":
Exception in thread "main" java.lang.NullPointerException
	at aser.ufo.misc.Addr2line.<init>(Addr2line.java:40)
	at aser.ufo.Session.init(Session.java:56)
	at aser.ufo.UFOMain.main(UFOMain.java:27)

I edited my config.properties to be
app_name=test
# trace directory to be analyzed
trace_dir=./ufo_traces
# solver timeout seconds
solver_time=120
#default symbolizer
symbolizer=atos
# solver_mem MB
solver_mem=2000
# window size
window_size=100000
# fast detect does not solve constraints to verify UAFs
fast_detect=false

and also tried changing the second line to trace_dir=./ufo_test_trace and trace_dir=./ufo_test_trace_22 to match the directory name of ufo_test_trace_22 in the ufo directory.

@qspark, is the variable trace_dir correctly set to the path of the directory of generated traces? For example, because of the way that the UFO directory is structured, the correct value should likely be ../ufo_test_trace_22.

Didn't see that I was missing a dot. I fixed the error by changing it to be trace_dir=../ufo_test_trace_22, being in the directory ufo-predict
and running java -jar ../ufo-predict/runufo.jar
Thanks!


Define ExperimentResults

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (30e2b8e) to head (fa4a0cc).
Report is 34 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##              main       #36   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            9        10    +1
  Lines          359       380   +21
=========================================
+ Hits           359       380   +21

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[CI] Enable python autoformat

Follow up to https://github.com/inventree/InvenTree/pull/6167. Enables auto-formatting of python files with pre-commit.

@SchrodingersGat I think this would be an improvement - what do you think?

Ready for review; adds autformating for python with ruff (similar to black - just a lot faster)

Please address conflicting files and I am happy to merge

@SchrodingersGat the code base is moving faaaaaast right now ;-)

@SchrodingersGat the code base is moving faaaaaast right now ;-)

Well, you have touched 13K lines of code 👍


Ubergraph

https://github.com/INCATools/ubergraph
Which parts of the functionality of ubergraph can we generalise and/or expose through OLS?
Can't just throw everything in OLS into the reasoner as would certainly be inconsistent and full of garbage. Maybe config option to specify lists of specific ontologies to turn into graphs and then a generalised API/UI over the top of them.

I would absolutely recommend against this. The ontology owners should supply the version of the ontology that they want the world to see, and there should be no reasoning, ubergraph or anything at all altering the structure of the ontology in graph form.

This would be a separate dataset. I’ll chat with you when I’m back from leave.

Just to jot some notes down from previous chats with @henrietteharmse
Currently OLS is backed by 2 Solr cores (1 for autocomplete, 1 for entity search) and 1 Neo4j graph.
We would like to enable more advanced DL-style queries over the OLS ontologies, which would be entirely separate from the core query functionality but accessible through the same API and UI. This would probably mean having some additional optional databases (perhaps e.g. a triplestore or another neo4j graph) which could maybe be populated using the same methods as ubergraph depending on how generalisable they are.

DL-style queries over the OLS ontologies

You can ask @dosumis who has been using https://owlery.docs.apiary.io/ in production in VFB pipeline - which may be one way, but not sure how performant that would be to load all of OLS. Probably not very.
Ubergraphs methods are entirely generalisable, we use it for all ontology as a proprocessing step in OAK (the main preprocessing is called "relation graph".
While an interesting avenue, I have had plenty of trouble with stuff like that in the past; I don't want to deter, just advice that approaching anything in this space will require significant resources (both developer and computational).

While an interesting avenue, I have had plenty of trouble with stuff like that in the past; I don't want to deter, just advice that approaching anything in this space will require significant resources (both developer and computational).

Absolutely - this is a research idea and perhaps we need a new issue label to separate it from things that can be immediately solved. Regardless the future potential of OLS is in exploiting the relationships captured by ontologies and exposing them as APIs; and currently ubergraph has that mindshare so it's exactly where we should be looking for ideas.


Regression of file loading outside of template location

The bug described in issue 3 has come back. Every version from 2.0.5 onwards now fails with the 'The template [../webinc/skeleton.pug] could not be opened. Maybe outside template path.' error.

Pug4J needs a Base path, so that it can determine the parent path. By default the basePath is empty.
To configure a basePath. You need to use the FileTemplateLoader or ClassPathTemplateLoader and configure setBase("").
Example for template location: /root/dir/base/path/template.pug
FileTemplateLoader fileLoader = new FileTemplateLoader("/root/dir/")
fileLoader.setBase("base/path")

PugConfiguration config = new PugConfiguration();
config.setTemplateLoader(fileLoader);
PugTemplate template = config.getTemplate("template");

Map<String, Object> model = new HashMap<String, Object>();

config.renderTemplate(template, model);

Pug is now allowed to read all files below /root/dir/ but not in / and /root
So this is more a security feature than a bug ;-) I will update the documentation.

You can't set a base path if you use Pug4J.render(path, model);. My expectation when using that call would be that there are no restrictions on which files you can point to.
There are many confusing things about the API (The code has a templateLoaderPath and a basePath and wants to load a template with a name that appears to just be a filename but without an extension?? You ask a config object to get and render templates? The rootpath must be absolute?), but the real issue is that your code example appears to just not work at all. With your code I now get a NoSuchFileException for a path that doesn't include the basepath. The code appears to ignore the basepath when looking for files.
Code is here. This is the error:
Exception in thread "main" java.nio.file.NoSuchFileException: C:\Projects\public\pug4j-bug\src\main\skeleton.pug
	at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
	at java.base/sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
	at java.base/sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
	at java.base/sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:197)
	at java.base/java.nio.file.Files.readAttributes(Files.java:1848)
	at java.base/java.nio.file.Files.getLastModifiedTime(Files.java:2391)
	at de.neuland.pug4j.template.FileTemplateLoader.getLastModified(FileTemplateLoader.java:65)
	at de.neuland.pug4j.PugConfiguration.getTemplate(PugConfiguration.java:57)
	at bug.Main.loadTemplate(Main.java:45)
	at bug.Main.main(Main.java:22)

Ok. I had the documentation wrong. What I found out, is you need to add a "/" in front of our "index" template name. This will select the base folder. Otherwise only the templateLoaderPath is used.
If you update your toBaseName method to the following code it should work:
    private static String toBaseName(final Path file) {
        final String filename = file.getFileName().toString();
        return "/"+filename.substring(0, filename.indexOf('.'));
    }

Then it prints out:
<title>Skeleton</title><link rel="stylesheet"<EMAIL_ADDRESS>rel="stylesheet" href="../webinc/style.css" inline="inline" compress="compress"/>

Yeah the forward slash works. The bug-repo is now out of date since the problem is solved so I'm going to set it as private.
Thanks.


Fix formatting of conditional types

Fixes #21716, pls review and let me know if there any changes required.

thanks!

Do you wana get the support for infer as well.. e.g.:
type C<T> = T extends Array<   infer     U  >? U : T;

@mhegazy I've created a test and it seems this is not related to conditional type formatting. The reason is there are no infer formatting rules defined yet.
How do I proceed? Do you want me to update this PR or are you going to create a new issue?
Code     : "type C<  T  >   =   T   extends   Array   <   infer     U  >   ;"
Expected : "type C<T> = T extends Array<infer U>;"
Actual   : "type C<T> = T extends Array<infer     U>   ;"

Code     : "type C<  T  >   =   T   extends   Array   <   infer     U  >  ? U : T;"
Expected : "type C<T> = T extends Array<infer U> ? U : T;"
Actual   : "type C<T> = T extends Array<infer     U> ? U : T;"

Do you know where can I find more details about infer keyword and examples?


status_server: add pprof flamegraph response header

Signed-off-by: Suhaha<EMAIL_ADDRESS>
What problem does this PR solve?
Problem Summary:
No related PR. The response of svg returned by the status server's pprof does not have a corresponding Content-Type header. And the correct return header makes it easier for different clients to handle the mime type.
What is changed and how it works?
What's Changed:
Add the correct Content-Type to response header.
Related changes

Need to cherry-pick to the release branch

Check List
Tests

Manual test

curl -v http://<IP_ADDRESS>:20180/debug/pprof/profile\?

will see the response header with Content-Type
< HTTP/1.1 200 OK
< content-type: image/svg+xml
< content-length: 187695
< date: Wed, 15 Sep 2021 15:49:13 GMT

Side effects
Release note

No release note

Sure, let me try.

I found a test case test_pprof_profile_service for pprof below, can I just add it here.

Please take another look @hunterlxt

/run-all-tests

/run-all-tests

/merge

/run-all-tests

/merge

/cc @5kbpers PTAL, thanks~

/merge

/run-all-tests

/run-all-tests

/merge

/merge

/merge


Minor nitpicking : provide a printable two-sided user's manual

The User's manual is provided as a PDF, which is compiled for single-sided printing. Whien printed two-sided, one gets a document where odd-sided pages are on the left page, some parts and chapter starting on a left page.
Ugly. For people used to usual printers' and bookbinders' conventions (a. k. a. old farts), this is hardly usable. And printing single-sided doubles the volume of the paper.
Two solutions :

Provide a two-sided PDF (with a \cleardoublepage  before each part and chapter), or
(better) add the \LaTeX source of the user's manual to the source that one can get from Github.

Furthermore, having the source allows to recompile for other paper sizes, or even to adapt font sizes and aim at a real booklet manual (more pages on a smaller booklet is easier to read and carry ; "booklet-printing" by scaling A4 to A5 causes type to be too small for easy use).
BTW: what fonts did you use for this ? Their rendering in some browser's PDF viewer is ... questionable.

Hi Emmanuel, these are great ideas and we welcome Pull Requests to the nimble-docs repo at https://github.com/nimble-dev/nimble-docs/tree/master/UserManual

Hi Emmanuel, these are great ideas and we welcome Pull Requests to the nimble-docs repo at https://github.com/nimble-dev/nimble-docs/tree/master/UserManual

Thanks. I didn't see this one...
I'll have a serious look at this. The fact that this is a .Rnw file is of great help. For now, the integration of the nimble source and the Manual source isn't very clear to me (BTW : why two trees rather than one ?) but opens a lot of possibilities.
If I think that my suggestions are useful, I'll open a pull request and a new issue.


Implement API call fluentWait analogue with timeout and polling interval

Sometimes we should do a lot of API calls  waiting for particular artifact in response.
For now it is organized on a project level and as side effect we do a lot of useless log_all_json logging operations.
So we suggest to implement API call response waiter with a timeout and polling interval and ability to log only last request and response into the log

done: https://github.com/zebrunner/carina/pull/1609
to be able to verify need to write/update any api tests in carina-demo: https://github.com/zebrunner/carina-demo/issues/217
@okamara, please review carina doc site in automation->api page #### Repeating API calling with the condition. actual changes you can find in above pull request

Verified, fixed.


choose a correct license

anyone can do anything - at least from my perspective
the policy is not "less strict" than any of the sources I've taken those words from

https://opensource.stackexchange.com/questions/15019/can-i-use-a-phrasal-verb-definition-from-a-certain-dictionary-in-my-mobile-app


Renamed SDGIShader to SDFGIShader and moved a bunch of things to private

Just a cleanup PR, renamed SDGIShader to SDFGIShader, that was a simple typo, and moved a bunch of structs and internal variables to private.
There is more to be made private but we can't do this until we've done some of the refactoring of the forward renderer in preparation of the mobile renderer but it's a good first pass and moving stuff from public to private.

Thanks!


Give some perspectives on our recruiting plan

Requires management approval

Approval obtained. Actually on the same slide as #313


Compare few-shot GPT4 features to embedding features for EFO term precision classification

@yonromai suggested trying to embed the text descriptions, labels, aliases, etc. associated with EFO terms and using those for embeddings as a part of https://github.com/related-sciences/nxontology-ml/issues/2.
It would be very interesting to see how a model like the one in https://github.com/related-sciences/nxontology-ml/pull/7 improves with embedding features by comparison to a model with only the few-shot labels in https://github.com/related-sciences/nxontology-ml/pull/6.
The LLM-derived features will definitely be harder to maintain/generate, but on the other hand I know the labels we provided in https://github.com/related-sciences/nxontology-ml/pull/5 are not perfect and I expect that the few-shot features will be more helpful in figuring out which ones are most likely to be mislabeled and why (since they can be directly compared).  Nevertheless, contrasting the predictive value of the two could potentially be an important determining factor for how this project, or at least #2 , evolves.

FYI @yonromai, this was the embedding model I had in mind when we last spoke: https://huggingface.co/michiyasunaga/BioLinkBERT-base.
That's from a top-tier group in the NLP space and it's the model submitted by the first author (michiyasunaga) on LinkBERT: Pretraining Language Models with Document Links (Mar. 2022).  The reported improvements on a recent SOTA model (PubMedBERT) are substantial, so it might be worth kicking the tires on it.
And to be clear, I have no allegiances to this over a LLaMA-derived model, OpenAI or some KG-based approach.  Any performance baseline using embeddings would be helpful.

cc: @eric-czech @dhimmel
TL;DR:

I have some code which builds additional features by embedding nodes' description using BioLinkBERT-base
The model performance improvement is currently insignificant (& the slowdown in terms of model training time is significant)
Little to no investigation has gone into (hyper)parameters of the model/features or performance analysis
I'd like to work a little on metrics (https://github.com/related-sciences/nxontology-ml/issues/9) before investigating/troubleshooting these new features
Experiment code is in this branch (not ready to merge)

Details:
Idea behind the new features:

At training time:

Each node's description is embedded using the BioLinkBERT-base
Each vector is used to build a Nearest Neighbors Tree (using Meta's Faiss)
The labels of each indexed node are saved for later retrieval

At inference time, for a given input node:

Embed the input node
Use the input node vector to fetch k-NN (k=50, not tuned)
Group the neighbors by label (from the training data)
Calculate, for each label, a set of descriptive statistics which summarize the cluster (the stats are support (i.e. cluster size), min, max, 1st quartile, median and 3rd quartile)

Outcome:

This approach is slow (~25min to run training + test set eval)
The gain in terms of performance is pretty much inexistent (see results below)

Comments:

Why using NN Trees?

I assumed that the embedding size would be too large to be exploited "as is" by the model (given that we have ~20k training points)
This tree approach is amongst the simplest I could come up with (but it's still more convoluted than I'd like it to be)

More analysis need to take place to understand if there is value in these new features (and whether the value added is worth the extra complexity)
More work on metrics (https://github.com/related-sciences/nxontology-ml/issues/9) and potentially experiment tracking (there are many Weight & Biases type of solutions in the OSS world) would be useful for ^

Experiment results:
With embedding features
Indexing Vectors: 100%|██████████| 17331/17331 [15:50<00:00, 18.40it/s]
Learning rate set to 0.091517
0:	learn: 1.2545993	total: 87.9ms	remaining: 1m 27s
100:	learn: 0.3817487	total: 1.74s	remaining: 15.4s
200:	learn: 0.3448022	total: 3.38s	remaining: 13.5s
300:	learn: 0.3226222	total: 5.06s	remaining: 11.7s
400:	learn: 0.3037497	total: 6.74s	remaining: 10.1s
500:	learn: 0.2870626	total: 8.47s	remaining: 8.43s
600:	learn: 0.2722362	total: 10.2s	remaining: 6.75s
700:	learn: 0.2571492	total: 11.9s	remaining: 5.07s
800:	learn: 0.2440707	total: 13.6s	remaining: 3.37s
900:	learn: 0.2329543	total: 15.3s	remaining: 1.68s
999:	learn: 0.2212473	total: 17s	remaining: 0us
> Feature importance:
                     Feature Id  Importances
0                        prefix    13.154424
1                    L4-support     8.085884
2                       n_roots     5.246222
3                    L2-support     4.897259
4          intrinsic_ic_sanchez     4.789902
5                    L1-support     4.645040
6                         depth     4.092159
7                   n_ancestors     3.673886
8   intrinsic_ic_sanchez_scaled     3.632503
9                        L2-min     3.356392
10                       L1-min     3.200558
11           xref__mondo__count     2.781731
12                   L3-support     2.633530
13                n_descendants     2.073415
14            xref__omim__count     1.899459
15                 intrinsic_ic     1.737778
16                       L3-min     1.673483
17                       L4-min     1.573646
18        xref__orphanet__count     1.524709
19            xref__ncit__count     1.478501
20          intrinsic_ic_scaled     1.456620
21                        L2-q1     1.455818
22                    n_parents     1.408397
23            xref__gard__count     1.368191
24            xref__doid__count     1.347312
25            xref__mesh__count     1.225176
26            xref__umls__count     1.107692
27          xref__omimps__count     1.101323
28                        L1-q1     0.880555
29            xref__icd9__count     0.803441
30                       L3-max     0.754765
31           xref__icd10__count     0.745556
32                       L1-med     0.744894
33                       L4-max     0.730284
34                   n_children     0.698697
35          xref__meddra__count     0.666259
36                       L2-max     0.643364
37        xref__snomedct__count     0.634071
38                        L3-q1     0.606200
39                       L3-med     0.597636
40                       L2-med     0.588662
41                        L4-q3     0.583918
42                is_gwas_trait     0.558833
43                        L2-q3     0.534196
44                        L3-q3     0.524524
45                        L4-q1     0.516574
46                       L1-max     0.450720
47                        L1-q3     0.402829
48                     n_leaves     0.377648
49                       L4-med     0.335366

> Classification report:
                    precision    recall  f1-score   support

01-disease-subtype       0.79      0.84      0.81      1085
   02-disease-root       0.70      0.66      0.68       797
   03-disease-area       0.79      0.72      0.76       257
    04-non-disease       0.98      0.98      0.98       920

          accuracy                           0.82      3059
         macro avg       0.82      0.80      0.81      3059
      weighted avg       0.82      0.82      0.82      3059

Without embedding features
Learning rate set to 0.091517
0:	learn: 1.2510844	total: 71.5ms	remaining: 1m 11s
100:	learn: 0.4182990	total: 1.06s	remaining: 9.43s
200:	learn: 0.3891503	total: 2.07s	remaining: 8.24s
300:	learn: 0.3713912	total: 3.05s	remaining: 7.09s
400:	learn: 0.3581384	total: 4.03s	remaining: 6.03s
500:	learn: 0.3451430	total: 5.01s	remaining: 4.99s
600:	learn: 0.3334726	total: 5.98s	remaining: 3.97s
700:	learn: 0.3224103	total: 7s	remaining: 2.98s
800:	learn: 0.3125697	total: 8.03s	remaining: 2s
900:	learn: 0.3042341	total: 9.04s	remaining: 993ms
999:	learn: 0.2964110	total: 10s	remaining: 0us
> Feature importance:
                     Feature Id  Importances
0                        prefix    19.235261
1                       n_roots     7.848109
2                   n_ancestors     7.547822
3                         depth     7.365341
4          intrinsic_ic_sanchez     5.770118
5            xref__mondo__count     4.362500
6             xref__omim__count     3.867349
7             xref__doid__count     3.414090
8         xref__orphanet__count     3.360891
9   intrinsic_ic_sanchez_scaled     3.086075
10            xref__mesh__count     3.016446
11                    n_parents     2.935589
12                is_gwas_trait     2.854420
13            xref__umls__count     2.633490
14            xref__ncit__count     2.608488
15                n_descendants     2.435195
16            xref__gard__count     2.336128
17                   n_children     2.183118
18                 intrinsic_ic     2.152569
19          xref__meddra__count     2.075885
20        xref__snomedct__count     1.854948
21           xref__icd10__count     1.849050
22          intrinsic_ic_scaled     1.791724
23            xref__icd9__count     1.153552
24                     n_leaves     1.142829
25          xref__omimps__count     1.119010

> Classification report:
                    precision    recall  f1-score   support

01-disease-subtype       0.80      0.85      0.82      1195
   02-disease-root       0.69      0.61      0.65       719
   03-disease-area       0.77      0.73      0.75       244
    04-non-disease       0.96      0.99      0.97       901

          accuracy                           0.82      3059
         macro avg       0.81      0.79      0.80      3059
      weighted avg       0.82      0.82      0.82      3059

Nice @yonromai!

More analysis need to take place to understand if there is value in these new features (and whether the value added is worth the extra complexity)

I think this is clear above, but that does not include GPT-4 assignments of the labels as features (i.e. from https://github.com/related-sciences/nxontology-ml/pull/6) correct?

I assumed that the embedding size would be too large to be exploited "as is" by the model

I think it would be ok to include the embeddings or a reduction on them (e.g. PCA) as features directly.  I like the tree/clustering approach, but my hunch is that it will be hard to show an improvement over that simpler method.

Experiment results

Do you have a sense of how much macro F1 averages vary across resamplings (e.g. with cross_val_score(..., cv=5, scoring='f1_macro'))?  It would be helpful to know what kind of performance loss in an ablation experiment like that should rate as substantial.

I think this is clear above, but that does not include GPT-4 assignments of the labels as features (i.e. from https://github.com/related-sciences/nxontology-ml/pull/6) correct?

That's right, I think we should do that next.

I think it would be ok to include the embeddings or a reduction on them (e.g. PCA) as features directly. I like the tree/clustering approach, but my hunch is that it will be hard to show an improvement over that simpler method.

Sure, I'll give it a try!

Do you have a sense of how much macro F1 averages vary across resamplings (e.g. with cross_val_score(..., cv=5, scoring='f1_macro'))? It would be helpful to know what kind of performance loss in an ablation experiment like that should rate as substantial.

That's a great question! I just started using ROC AUC & MAE's from #9 to look into the performance of the model. I'll spend a little bit of time in notebook land looking at how features & model parameters influences metrics.

looking at how features & model parameters influences metrics

Awesome, sounds good!  So we're clear though, I'm proposing that we compare distributions of F1, ROC, MAE, etc. scores between models where the distributions come from multiple evaluations of those metrics for different folds.  Given that this dataset is small, I think we'll need that help understand what changes are significant.  Would you agree?

Yes totally agree, the idea is to "repeat the experiment" of training the same model on different (stratified) folds of the training set to get an idea of the spread of metrics. Then we can use this spread to have an idea of the significance of the metrics calculated once we change the model/features. Is that what you meant?

Okay so after some time in notebook land here is gist of what I found out:
TL;DR
@eric-czech both of your hunches were 💯 :

There is quite a bit of metrics variation when training the same model => training on multiple folds helps
The best way to leverage the node text embeddings is to do PCA on the embeddings (as opposed to doing KNN)

@dhimmel Implementing the MAE (with the class biases suggested by @eric-czech) has proven very useful!
Some results:

For comparison:

A random classifier with real class weights prior has a biased MAE of 0.284
A random classifier with uniform class weights prior has a biased MAE of 0.348

More details about the best performing model (/ Food for thoughts)
The model seems to max out on BiasedMAE on the training data (not on the eval/CV one):

The model seems to slightly overfit the objective function:

More details..
For more details about the experiments & findings, take a look at the notebook. All the (non-production ready) code is in my branch.

@eric-czech I think now I have enough understanding about the performance of the pre-GPT model that it'd be worth running the training data through the GPT4 prompt you provided and see if that does better than the model out of the box!
I can run some estimation of the cost of the procedure if useful.

here is gist of what I found out

Very nice @yonromai!  Great experimental setup and it's excellent to see some clear separation between those models.

Some results:

For posterity, I think it would be helpful to say more about what the lda_d7 and knn_d7 configurations were at this level.  Presumably knn_d7 was the method in https://github.com/related-sciences/nxontology-ml/issues/8#issuecomment-1679531083.  What was lda_d7 though?
Noting the current details in the notebook:

running the training data through the GPT4 prompt you provided and see if that does better than the model out of the box!

Awesome -- I'd love to see how it performs on its own and when included as a feature with the other baseline features in a catboost gbm.

More details about the best performing model

OOC what is that UI you're looking at there?  I don't see any obvious hints in https://github.com/related-sciences/nxontology-ml/tree/romain/embeddings/experimentation.

For posterity, I think it would be helpful to say more about what the lda_d7 and knn_d7 configurations were at this level. Presumably knn_d7 was the method in https://github.com/related-sciences/nxontology-ml/issues/8#issuecomment-1679531083. What was lda_d7 though?

@eric-czech Noted, I'll add more details in the notebook. (The LDA code directly applies Sklearn's LDA, similar to the PCA - see this code)
I'd like to cleanup the code I have in my branch and merge it into the main branch. I'm probably going to end up deleting a lot of the code (e.g. the KNN part) in the near future but that way it'll be saved in git history (along with the experimental setup).
@dhimmel: Would that fine with you? (It's gonna be quite a big PR :( )

OOC what is that UI you're looking at there? I don't see any obvious hints in https://github.com/related-sciences/nxontology-ml/tree/romain/embeddings/experimentation.

The code which displays the model metrics is in the "CatBoost's MetricVisualizer" section of the notebook but it's JavaScript so it doesn't render in GH.

The code which displays the model metrics is in the "CatBoost's MetricVisualizer" section of the notebook but it's JavaScript so it doesn't render in GH.

Sometimes this will render in nbviewer, but not in this case.

Are there short terms plans to work on this or is it appropriate to close this issue?

Given that GPT assignments were inferior to text embedding features and didn't add much when combined, I don't think we need to use GPT features at all. Saves on cost and complexity.

I don't think we need to use GPT features at all

I definitely agree.  Noting https://github.com/related-sciences/nxontology-ml/pull/34#issue-1912661241 as the most recent experiment at TOW that still had these features.


Deploy a CAP Business Application to SAP Business Technology Platform

Tutorials: https://developers.sap.com/tutorials/cap-service-deploy.html
Issue: The deployed application my-bookshop-srv is not able to load the data
Description: In step 3.3 After we deploy the application, it is able to launch the application but not able to find the table i.e if we click on Authors getting the "Internal server error"

Logs:
C:\Users\C5309305\my-bookshop>cf logs my-bookshop-srv --recent
Retrieving logs for app my-bookshop-srv in org 4997ea4dtrial / space dev as<EMAIL_ADDRESS>2021-06-11T15:02:02.84+0530 [API/122] OUT Created app with guid 42c315a7-040d-4c69-8772-04605c103c8b
2021-06-11T15:02:05.42+0530 [API/82] OUT Uploading bits for app with guid 42c315a7-040d-4c69-8772-04605c103c8b
2021-06-11T15:02:12.27+0530 [API/71] OUT Creating build for app with guid 42c315a7-040d-4c69-8772-04605c103c8b
2021-06-11T15:02:12.55+0530 [API/71] OUT Updated app with guid 42c315a7-040d-4c69-8772-04605c103c8b ({"state"=>"STARTED"})
2021-06-11T15:02:12.80+0530 [STG/0] OUT Downloading nodejs_buildpack...
2021-06-11T15:02:12.83+0530 [STG/0] OUT Downloaded nodejs_buildpack
2021-06-11T15:02:12.83+0530 [STG/0] OUT Cell d52c9115-37a6-473b-847d-4f8b095bd6a5 creating container for instance 9339efe8-82dc-4e38-89ae-be14c2027f8e
2021-06-11T15:02:14.47+0530 [STG/0] OUT Cell d52c9115-37a6-473b-847d-4f8b095bd6a5 successfully created container for instance 9339efe8-82dc-4e38-89ae-be14c2027f8e
2021-06-11T15:02:14.64+0530 [STG/0] OUT Downloading app package...
2021-06-11T15:02:14.69+0530 [STG/0] OUT Downloaded app package (10K)
2021-06-11T15:02:14.95+0530 [STG/0] OUT -----> Nodejs Buildpack version 1.7.48
2021-06-11T15:02:14.96+0530 [STG/0] OUT -----> Installing binaries
2021-06-11T15:02:14.96+0530 [STG/0] OUT        engines.node (package.json): >=12
2021-06-11T15:02:14.96+0530 [STG/0] OUT        engines.npm (package.json): unspecified (use default)
2021-06-11T15:02:14.96+0530 [STG/0] OUT        WARNING Dangerous semver range (>) in engines.node.   Please see the prereqs for SAP HANA Cloud service and updated steps (especially 3.1) in the tutorial.
Best regards,
Iwona

Hi @iwonahahn
Still having issues getting the data displayed.

Logs
C:\Users\C5309305\my-bookshop>cf logs my-bookshop-srv
Retrieving logs for app my-bookshop-srv in org 266b2262trial / space dev as<EMAIL_ADDRESS>2021-07-10T00:23:38.02+0530 [STG/0] OUT added 193 packages from 176 contributors and audited 303 packages in 5.614s   2021-07-10T00:23:38.10+0530 [STG/0] OUT 33 packages are looking for funding
2021-07-10T00:23:38.10+0530 [STG/0] OUT   run npm fund for details
2021-07-10T00:23:38.10+0530 [STG/0] OUT found 0 vulnerabilities
2021-07-10T00:23:38.76+0530 [STG/0] OUT        WARNING Unmet dependencies don't fail npm install but may cause
runtime issues
2021-07-10T00:23:38.76+0530 [STG/0] OUT        See: https://github.com/npm/npm/issues/7494
2021-07-10T00:23:38.77+0530 [STG/0] OUT        Contrast Security no credentials found.  Existing pool will be drained.
2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR [cds] - { TimeoutError: Acquiring client from pool timed out. Please review your system setup, transaction handling, and pool configuration.
2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR     at ResourceRequest._fireTimeout (/home/vcap/deps/0/node_modules/@sap/cds-foss/node_modules/generic-pool/lib/ResourceRequest.js:62:17)
2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR     at Timeout.bound (/home/vcap/deps/0/node_modules/@sap/cds-foss/node_modules/generic-pool/lib/ResourceRequest.js:8:15)
2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR     at ontimeout (timers.js:436:11)
2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR     at tryOnTimeout (timers.js:300:5)
2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR     at listOnTimeout (timers.js:263:5)
2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR     at Timer.processTimers (timers.js:223:10)
2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR   name: 'TimeoutError',
2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR   statusCode: 503,
2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR   id: '1615004',
2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR   level: 'ERROR',
2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR   timestamp:<PHONE_NUMBER>008 }
2021-07-10T00:24:42.06+0530 [RTR/23] OUT my-bookshop-srv-busy-hyena-ku.cfapps.eu10.hana.ondemand.com - [2021-07-09T18:54:40.654694145Z] "GET /catalog/Authors?$top=11 HTTP/1.1" 503 0 160 "https://my-bookshop-srv-busy-hyena-ku.cfapps.eu10.hana.ondemand.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36" "-" "<IP_ADDRESS>:61033" x_forwarded_for:"-" x_forwarded_proto:"https" vcap_request_id:"e1f6d9e7-09a6-4420-4be6-2c609edb7af4" response_time:1.408848 gorouter_time:0.034545 app_id:"fce92d6d-fa02-424f-abb2-d8771ab96bf4" app_index:"0" x_cf_routererror:"-" x_correlationid:"-" tenantid:"-" sap_passport:"-" x_scp_request_id:"5187493b-23df-4861-b35f-c5cc583d6385-60E89B70-26A569" x_cf_app_instance:"-" x_forwarded_host:"-" x_custom_host:"-" x_b3_traceid:"cb5915366e5e8494" x_b3_spanid:"cb5915366e5e8494" x_b3_parentspanid:"-" b3:"cb5915366e5e8494-cb5915366e5e8494"
2021-07-10T00:24:42.06+0530 [RTR/23] OUT
2021-07-10T00:27:37.49+0530 [APP/PROC/WEB/0] ERR [cds] - GET /catalog/Authors?$top=11
2021-07-10T00:27:37.49+0530 [APP/PROC/WEB/0] ERR [cds.hana] - effective pool configuration: { min: 0,
2021-07-10T00:27:37.49+0530 [APP/PROC/WEB/0] ERR   max: 100,
2021-07-10T00:27:37.49+0530 [APP/PROC/WEB/0] ERR   testOnBorrow: true,
2021-07-10T00:27:37.49+0530 [APP/PROC/WEB/0] ERR   acquireTimeoutMillis: 1000,
2021-07-10T00:27:37.49+0530 [APP/PROC/WEB/0] ERR   softIdleTimeoutMillis: 30000,
2021-07-10T00:27:37.49+0530 [APP/PROC/WEB/0] ERR   idleTimeoutMillis: 30000,
2021-07-10T00:27:37.49+0530 [APP/PROC/WEB/0] ERR   evictionRunIntervalMillis: 60000,
2021-07-10T00:27:37.49+0530 [APP/PROC/WEB/0] ERR   numTestsPerEvictionRun: 34 }
2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR [cds.hana] - { Error: Could not establish connection for tenant "anonymous" due to error: Could not connect to any host: [ 3c31055d-23d6-460c-a439-ca3c9bea5e8e.hana.trial-eu10.hanacloud.ondemand.com:443 - Client network socket disconnected before secure TLS connection was established ]
2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR     at couldNotOpenConnectionError (/home/vcap/deps/0/node_modules/hdb/lib/protocol/ConnectionManager.js:115:13)
2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR     at tryConnect (/home/vcap/deps/0/node_modules/hdb/lib/protocol/ConnectionManager.js:35:17)
2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR     at /home/vcap/deps/0/node_modules/hdb/lib/protocol/ConnectionManager.js:42:9
2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR     at TLSSocket.onerror (/home/vcap/deps/0/node_modules/hdb/lib/protocol/Connection.js:175:5)
2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR     at Object.onceWrapper (events.js:286:20)
2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR     at TLSSocket.emit (events.js:198:13)
2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR     at emitErrorNT (internal/streams/destroy.js:91:8)
2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR     at process._tickCallback (internal/process/next_tick.js:63:19) code: 'EHDBOPENCONN' }
2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR [cds.hana] - Could not establish connection for tenant "anonymous". Existing pool will be drained.
2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR [cds] - { TimeoutError: Acquiring client from pool timed out. Please review your system setup, transaction handling, and pool configuration.
2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR     at ResourceRequest._fireTimeout (/home/vcap/deps/0/node_modules/@sap/cds-foss/node_modules/generic-pool/lib/ResourceRequest.js:62:17)
2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR     at Timeout.bound (/home/vcap/deps/0/node_modules/@sap/cds-foss/node_modules/generic-pool/lib/ResourceRequest.js:8:15)
2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR     at ontimeout (timers.js:436:11)
2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR     at tryOnTimeout (timers.js:300:5)
2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR     at listOnTimeout (timers.js:263:5)
2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR     at Timer.processTimers (timers.js:223:10)
2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR   name: 'TimeoutError',
2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR   statusCode: 503,
2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR   id: '1615004',
2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR   level: 'ERROR',
2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR   timestamp:<PHONE_NUMBER>498 }
2021-07-10T00:27:38.50+0530 [RTR/3] OUT my-bookshop-srv-busy-hyena-ku.cfapps.eu10.hana.ondemand.com - [2021-07-09T18:57:37.481601564Z] "GET /catalog/Authors?$top=11 HTTP/1.1" 503 0 160 "https://my-bookshop-srv-busy-hyena-ku.cfapps.eu10.hana.ondemand.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36" "-" "<IP_ADDRESS>:61033" x_forwarded_for:"-" x_forwarded_proto:"https" vcap_request_id:"61ca4c9a-49d9-44a5-63f0-ab2ee6ae74a3" response_time:1.018562 gorouter_time:0.000121 app_id:"fce92d6d-fa02-424f-abb2-d8771ab96bf4" app_index:"0" x_cf_routererror:"-" x_correlationid:"-" tenantid:"-" sap_passport:"-" x_scp_request_id:"cc8edb65-9eec-4d68-98fa-557f8273055c-60E89C20-2F96E5" x_cf_app_instance:"-" x_forwarded_host:"-" x_custom_host:"-" x_b3_traceid:"48d3964a2b33c337" x_b3_spanid:"48d3964a2b33c337" x_b3_parentspanid:"-" b3:"48d3964a2b33c337-48d3964a2b33c337"
2021-07-10T00:27:38.50+0530 [RTR/3] OUT
Labels:
Landscape Labels: trail1, trail2
Scenario Label: SCPTest-banode
Priority: High
Sprint Label: SCPTest-2107A

Hi @Askullu,
Your SAP HANA Cloud instance is up and running? I will try the tutorial with the latest CAP release and will get back to you asap.
Best regards,
Iwona

Hi @iwonahahn
we have the exact same issue...we first developed locally with hana-cli  "@sap/hana-client": "^2.9.28", but after deployment to CF, we got that timeout error mentioned above. It seems that the latest hana-cli version used on CF is responsible and leads to these side effects. I could reproduce this issue locally when trying hana-cli "@sap/hana-client": "^2.10.13",...Do you have any idea how to solve this issue?
Message even during cds watch:
hana] - Could not establish connection for tenant "anonymous". Existing pool will be drained.
[cds] - TimeoutError: Acquiring client from pool timed out. Please review your system setup, transaction handling, and pool configuration.
Thank you.
BR
Rufat

hey folks, any update on this? @NidhiMurkute @iwonahahn

Hi @maxstreifeneder,
The tutorial works as it is, also tested twice by us and the testing team. Nevertheless, we plan to update the tutorial using the Cloud MTA Build Tool as described in the CAP documentation.
Best regards,
Iwona

Hi,
I am looking for some pointers and help as I am continuing to have issues while trying to deploy MTA Archives. Everything is fine till "cds watch --profile hybrid". The command "mbt build" also works fine. The moment I try to deploy it it gives an error.
Package.json
{   "name": "e_learning",   "version": "1.0.0",   "description": "A simple CAP project.",   "repository": "<Add your repository here>",   "license": "UNLICENSED",   "engines": {     "node": "^18"   },   "files": [     "db",     "srv"   ],   "xdependencies": {     "@sap/fesr-to-otel-js": <EMAIL_ADDRESS> },   "private": true,   "dependencies": {     "@sap/cds": "^7.2.0",     "@sap/cds-hana": "^2",     "express": "^4",     "passport": "^0.7.0"   },   "devDependencies": {     "@cap-js/sqlite": "^1",     "@sap/cds-dk": "^7",     "sqlite3": "5.0.0"   },   "scripts": {     "start": "cds-serve"   },   "sapux": [     "app/e_learning"   ],   "cds": {     "build": {       "tasks": [         {           "for": "hana",           "dest": "../db",           "options": {             "model": [               "db",               "srv"             ]           }         },         {           "src": "srv",           "for": "node-cf",           "options": {             "model": [               "db",               "srv"             ]           }         }       ]     },     "requires": {       "db": {         "kind":"hana-cloud"        }     }   } }
mta.yaml
`_schema-version: '3.1'
ID: e_learning
version: 1.0.0
description: A simple CAP project.
parameters:
enable-parallel-deployments: true
build-parameters:
before-all:
- builder: custom
commands:
# - npm install --production
# - npx -p @sap/cds-dk cds build --production
- npx cds build --production
modules:

name: e_learning-srv
type: nodejs
path: gen/srv
parameters:
buildpack: nodejs_buildpack
memory: 512M
disk-quota: 1024M
command: node ./node_modules/@sap/cds/bin/cds-serve
properties:
OPTIMIZE_MEMORY: true
build-parameters:
builder: npm
provides:

name: srv-api
properties:
srv-url: '${default-url}'
requires:
name: e_learning-db

name: e_learning-db-deployer
type: hdb
path: db
parameters:
buildpack: nodejs_buildpack
requires:

name: e_learning-db
resources:

name: e_learning-db
type: com.sap.xs.hdi-container
parameters:
service: hana
service-plan: hdi-shared
`

I have looked at the logs and I can see that error in db-deployer.log file
#2.0#2023 12 25 17:30:53.193#Z#DEBUG#com.sap.cloud.lm.sl.xs2.461d3d28-a34b-11ee-b808-eeee0a8c7d60.e_learning-db-deployer.stageAppTask# ######org.cloudfoundry.multiapps.controller.persistence.services.ProcessLogger########flowable-async-job-executor-thread-354### [PollStageAppStatusExecution] [e_learning-db-deployer] e1336439-240b-4c0b-90e4-aac332660e1d [2023-12-25T17:25:22.011848] NPM_CONFIG_LOGLEVEL=error (STDOUT, STG)#
in -srv.log
#2.0#2023 12 25 17:30:53.183#Z#DEBUG#com.sap.cloud.lm.sl.xs2.461d3d28-a34b-11ee-b808-eeee0a8c7d60.e_learning-srv.stageAppTask# ######org.cloudfoundry.multiapps.controller.persistence.services.ProcessLogger########flowable-async-job-executor-thread-355### [PollStageAppStatusExecution] [e_learning-srv] 14cb10fc-bdc0-40a2-9793-ef4671b99232 [2023-12-25T17:25:19.534446] NPM_CONFIG_LOGLEVEL=error (STDOUT, STG)#

Hi @iwonahahn Still having issues getting the data displayed.  Logs C:\Users\C5309305\my-bookshop>cf logs my-bookshop-srv Retrieving logs for app my-bookshop-srv in org 266b2262trial / space dev as<EMAIL_ADDRESS>2021-07-10T00:23:38.02+0530 [STG/0] OUT added 193 packages from 176 contributors and audited 303 packages in 5.614s 2021-07-10T00:23:38.10+0530 [STG/0] OUT 33 packages are looking for funding 2021-07-10T00:23:38.10+0530 [STG/0] OUT run npm fund for details 2021-07-10T00:23:38.10+0530 [STG/0] OUT found 0 vulnerabilities 2021-07-10T00:23:38.76+0530 [STG/0] OUT WARNING Unmet dependencies don't fail npm install but may cause runtime issues 2021-07-10T00:23:38.76+0530 [STG/0] OUT See: npm/npm#7494 2021-07-10T00:23:38.77+0530 [STG/0] OUT Contrast Security no credentials found.  Existing pool will be drained. 2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR [cds] - { TimeoutError: Acquiring client from pool timed out. Please review your system setup, transaction handling, and pool configuration. 2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR at ResourceRequest._fireTimeout (/home/vcap/deps/0/node_modules/@sap/cds-foss/node_modules/generic-pool/lib/ResourceRequest.js:62:17) 2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR at Timeout.bound (/home/vcap/deps/0/node_modules/@sap/cds-foss/node_modules/generic-pool/lib/ResourceRequest.js:8:15) 2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR at ontimeout (timers.js:436:11) 2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR at tryOnTimeout (timers.js:300:5) 2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR at listOnTimeout (timers.js:263:5) 2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR at Timer.processTimers (timers.js:223:10) 2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR name: 'TimeoutError', 2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR statusCode: 503, 2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR id: '1615004', 2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR level: 'ERROR', 2021-07-10T00:24:42.00+0530 [APP/PROC/WEB/0] ERR timestamp:<PHONE_NUMBER>008 } 2021-07-10T00:24:42.06+0530 [RTR/23] OUT my-bookshop-srv-busy-hyena-ku.cfapps.eu10.hana.ondemand.com - [2021-07-09T18:54:40.654694145Z] "GET /catalog/Authors?$top=11 HTTP/1.1" 503 0 160 "https://my-bookshop-srv-busy-hyena-ku.cfapps.eu10.hana.ondemand.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36" "-" "<IP_ADDRESS>:61033" x_forwarded_for:"-" x_forwarded_proto:"https" vcap_request_id:"e1f6d9e7-09a6-4420-4be6-2c609edb7af4" response_time:1.408848 gorouter_time:0.034545 app_id:"fce92d6d-fa02-424f-abb2-d8771ab96bf4" app_index:"0" x_cf_routererror:"-" x_correlationid:"-" tenantid:"-" sap_passport:"-" x_scp_request_id:"5187493b-23df-4861-b35f-c5cc583d6385-60E89B70-26A569" x_cf_app_instance:"-" x_forwarded_host:"-" x_custom_host:"-" x_b3_traceid:"cb5915366e5e8494" x_b3_spanid:"cb5915366e5e8494" x_b3_parentspanid:"-" b3:"cb5915366e5e8494-cb5915366e5e8494" 2021-07-10T00:24:42.06+0530 [RTR/23] OUT 2021-07-10T00:27:37.49+0530 [APP/PROC/WEB/0] ERR [cds] - GET /catalog/Authors?$top=11 2021-07-10T00:27:37.49+0530 [APP/PROC/WEB/0] ERR [cds.hana] - effective pool configuration: { min: 0, 2021-07-10T00:27:37.49+0530 [APP/PROC/WEB/0] ERR max: 100, 2021-07-10T00:27:37.49+0530 [APP/PROC/WEB/0] ERR testOnBorrow: true, 2021-07-10T00:27:37.49+0530 [APP/PROC/WEB/0] ERR acquireTimeoutMillis: 1000, 2021-07-10T00:27:37.49+0530 [APP/PROC/WEB/0] ERR softIdleTimeoutMillis: 30000, 2021-07-10T00:27:37.49+0530 [APP/PROC/WEB/0] ERR idleTimeoutMillis: 30000, 2021-07-10T00:27:37.49+0530 [APP/PROC/WEB/0] ERR evictionRunIntervalMillis: 60000, 2021-07-10T00:27:37.49+0530 [APP/PROC/WEB/0] ERR numTestsPerEvictionRun: 34 } 2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR [cds.hana] - { Error: Could not establish connection for tenant "anonymous" due to error: Could not connect to any host: [ 3c31055d-23d6-460c-a439-ca3c9bea5e8e.hana.trial-eu10.hanacloud.ondemand.com:443 - Client network socket disconnected before secure TLS connection was established ] 2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR at couldNotOpenConnectionError (/home/vcap/deps/0/node_modules/hdb/lib/protocol/ConnectionManager.js:115:13) 2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR at tryConnect (/home/vcap/deps/0/node_modules/hdb/lib/protocol/ConnectionManager.js:35:17) 2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR at /home/vcap/deps/0/node_modules/hdb/lib/protocol/ConnectionManager.js:42:9 2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR at TLSSocket.onerror (/home/vcap/deps/0/node_modules/hdb/lib/protocol/Connection.js:175:5) 2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR at Object.onceWrapper (events.js:286:20) 2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR at TLSSocket.emit (events.js:198:13) 2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR at emitErrorNT (internal/streams/destroy.js:91:8) 2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR at process._tickCallback (internal/process/next_tick.js:63:19) code: 'EHDBOPENCONN' } 2021-07-10T00:27:37.50+0530 [APP/PROC/WEB/0] ERR [cds.hana] - Could not establish connection for tenant "anonymous". Existing pool will be drained. 2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR [cds] - { TimeoutError: Acquiring client from pool timed out. Please review your system setup, transaction handling, and pool configuration. 2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR at ResourceRequest._fireTimeout (/home/vcap/deps/0/node_modules/@sap/cds-foss/node_modules/generic-pool/lib/ResourceRequest.js:62:17) 2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR at Timeout.bound (/home/vcap/deps/0/node_modules/@sap/cds-foss/node_modules/generic-pool/lib/ResourceRequest.js:8:15) 2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR at ontimeout (timers.js:436:11) 2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR at tryOnTimeout (timers.js:300:5) 2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR at listOnTimeout (timers.js:263:5) 2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR at Timer.processTimers (timers.js:223:10) 2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR name: 'TimeoutError', 2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR statusCode: 503, 2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR id: '1615004', 2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR level: 'ERROR', 2021-07-10T00:27:38.49+0530 [APP/PROC/WEB/0] ERR timestamp:<PHONE_NUMBER>498 } 2021-07-10T00:27:38.50+0530 [RTR/3] OUT my-bookshop-srv-busy-hyena-ku.cfapps.eu10.hana.ondemand.com - [2021-07-09T18:57:37.481601564Z] "GET /catalog/Authors?$top=11 HTTP/1.1" 503 0 160 "https://my-bookshop-srv-busy-hyena-ku.cfapps.eu10.hana.ondemand.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36" "-" "<IP_ADDRESS>:61033" x_forwarded_for:"-" x_forwarded_proto:"https" vcap_request_id:"61ca4c9a-49d9-44a5-63f0-ab2ee6ae74a3" response_time:1.018562 gorouter_time:0.000121 app_id:"fce92d6d-fa02-424f-abb2-d8771ab96bf4" app_index:"0" x_cf_routererror:"-" x_correlationid:"-" tenantid:"-" sap_passport:"-" x_scp_request_id:"cc8edb65-9eec-4d68-98fa-557f8273055c-60E89C20-2F96E5" x_cf_app_instance:"-" x_forwarded_host:"-" x_custom_host:"-" x_b3_traceid:"48d3964a2b33c337" x_b3_spanid:"48d3964a2b33c337" x_b3_parentspanid:"-" b3:"48d3964a2b33c337-48d3964a2b33c337" 2021-07-10T00:27:38.50+0530 [RTR/3] OUT
Labels: Landscape Labels: trail1, trail2 Scenario Label: SCPTest-banode Priority: High Sprint Label: SCPTest-2107A

@Askullu did you manage to resolve this issue? If yes, could you please let me know as to how?
<EMAIL_ADDRESS>

@smahati can you please have a look here?
@shishirp05 I hope @smahati can help you here.

Hi,
It seems like there is an issue with the database connectivity as per the logs
effective pool configuration: { min: 0, 2021-07-10T00:24:41.00+0530 [APP/PROC/WEB/0] ERR max: 100, 2021-07-10T00:24:41.00+0530 [APP/PROC/WEB/0] ERR testOnBorrow: true, 2021-07-10T00:24:41.00+0530 [APP/PROC/WEB/0] ERR acquireTimeoutMillis: 1000, 2021-07-10T00:24:41.00+0530 [APP/PROC/WEB/0] ERR softIdleTimeoutMillis: 30000, 2021-07-10T00:24:41.00+0530 [APP/PROC/WEB/0] ERR idleTimeoutMillis: 30000, 2021-07-10T00:24:41.00+0530 [APP/PROC/WEB/0] ERR evictionRunIntervalMillis: 60000, 2021-07-10T00:24:41.00+0530 [APP/PROC/WEB/0] ERR numTestsPerEvictionRun: 34 }
I will check with the topic owners to get their input and add it here.
Best regards,
Mahati


Amend parami protocol

#160

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: DorianRust:x: hellotrongoYou have signed the CLA already but the status is still pending? Let us recheck it.

Thanks for the amendment @hellotrongo. As I mentioned, could you please include deliverables 0a-d as in the template in all milestones, and 0e at least in the last one? Also I think the price should be split evenly across all milestones. You're quoting a similar workload for each of them and if anything, it seems to me the last milestone would be the most extensive one.

Hello @alxs , thanks for the suggestions. I've updated the pr.

Hi @alx, sorry to disturb you again. The cost has been adjusted. Besides this part, I believe I have amended everything based on your suggestion. What else did I miss? Thank you.

Hi Dorian. Could you please include the same deliverables 0a-d as in the template in every milestone? Eventually you can change the wording etc. But they should be in the spirit of those in the template i.e. 0a is the license, 0b is documentation etc. These are "meta-deliverables" which is why they are labelled with 0. Your other deliverables should each have their own number and can be split into a, b, c, if they clearly belong together or are part of the same component.

Hi @alxs, thanks for your suggestions. I have revised the PR accordingly.

Thanks, could you label the deliverables according to what they are? I.e. instead of 10 times "substrate chain" please name the pallet(s)/interface that you will develop. The rest looks good to me.

Hi @alxs, Hi, please kindly find the amended version. Thanks for all the suggestions and support.

Sorry @hellotrongo, I'm gonna go through it one more time. The new deliverables diverge considerably from the original application.

Hi @alxs , Thanks for your support along the way. Given that I received very informative suggestions on your side last time, I dived into the deliverables and reconstructed the whole stuff. The current version is more concise because I removed some unnecessary parts based on your suggestions. If any further advice, please feel free to contact me. Thanks.

As you will. The delivery would still be overdue with the updated duration, could you calculate it from the date your original application was accepted or just include an ETA? Thanks.

Hi @alxs, thanks for the reminder. The deadline is updated to until December, which would be 12 months since the original application was accepted.

@hellotrongo thanks. That's a very long timeframe, I personally think in that case and seeing as your roadmap doesn't seem to be exactly aligned with the deliverables, it would make more sense for everyone involved to terminate the current grant and that you submit a new application. Let's see what the rest of the committee thinks.

Hi @alxs. It’s basically same, I just add a little bit more content. And we already finished some content of grant, why do you terminate my grant?

Hi @alxs, We didn’t change the basic structure, only to make some minor adjustments and a few additions. Given the only major change is time duration alone, do we have to submit a new application? We’d be happy to re-apply, while it would be better if we can make amendments to the original one.

Thanks. In this case, my recommendation is also to cancel the current contract and potentially sign a new contract at the end of the year. A lot of things might change in half a year, especially in the crypto space. For example other teams might already have implemented the features, or you might no longer want to work on this, etc.

Hi @alxs, After some careful evaluation with the tech team, we found that 8-month duration is doable. I have amended the duration here. Thanks.

#160

@hellotrongo it seems the rest of the committee isn't convinced by the amendment either, despite the shortened duration. I'm closing the PR and putting the grant forward for termination again (which we shouldn't have undone without negotiating the amendment first, apologies for the oversight). As mentioned, feel free to apply again with your current roadmap.


Change default, options, and explanation for issue priority

Right now, we get a lot of "P0s" because people don't change the default or don't understand the priority meanings. In all templates, this:

Reverses the dropdown order so that p3 is the default.
Adds a link to the issue priorities doc.
Removes P0 from the feature/task templates since those are definitionally not P0 (or at the very least should only be set that way by people with label triage permission).

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[x] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Can one of the admins verify this patch?


Initial C# conversion

@coding-horror I have no idea why the CI fails, but it's for sure not related to this PR.

Ok. Thanks!  I'm not the best with Git yet so was worried I did something wrong.

Looks good thank you!


Other code cannot be run

Hello,
I'm using this Node Module as a benefit for a DiscordJS client, and the other actions cannot be run while the prompt is activated.
How can I run the script and the readline in the same time?
I'm actually using promptCL like this:
readlineSync.promptCLLoop({
    shutdown: () => {
        console.log("Shutdown the bot client.");
        process.exit(0);
    }
})

bot.login(config.token.discord); // Connection method from Discord

I don't think I'm the only one who have this kind of issue, it's why I've created a issue ticket.
Best regards,
LightOxygen

Hi @OPS-LightOxygen, thank you for the comment.
The promptCLLoop executes promptCL method repeatedly.
Therefore, the line including bot.login... is run after the promptCLLoop exited from the loop.
See document: https://github.com/anseki/readline-sync#promptclloop
I don't know what you want to do, but maybe your code should be like this:
readlineSync.promptCLLoop({
  login() {
    bot.login(config.token.discord); // Connection method from Discord
    // Or, login before promptCLLoop
  }
  shutdown() { return true; }

  // Other commands...
  message(text) {
    message.reply(text);
  }
});

console.log('Shutdown the bot client.');
// Closing process...

Hi,
Thanks for your reply.
Have a nice day,

:smile:


Circular types?

How would you handle something like the following, as far ordering the types? Thanks!
type Comment {
  user: User
  comment: String!
  id: ID!
}

type User {
  comments: [Comment!]!
  id: ID!
}

This is the current limitation I have which I'm not sure how to solve it yet as TypeScript is eagerly typed.
But if really need, you need to manually type it yourself for now.
type Comment = {
    comment: string
    id: string
    user: User
}

type User = {
    comments: [Comment]
    id: string
}

const client = new Mobius<typeof typeDefs, {
    User: User
    Comment: Comment
}>()

Please keep this opened until I find a way to solve this

I saw this project on twitter and thought it looked really cool so I tried investigating a proof of concept on how to solve this circular types issue.
I ended up coming up with this: https://tsplay.dev/mLX7vm
It uses the fact that interfaces can reference the this type to have recursion. I haven't yet tried to integrate it with the rest of mobius but the concept seems to work well.
I'd be happy to try to create a PR to make it work

Thanks to @ggrandi implementation, should be fixed with #2.
Please update to 0.1.0 to test it out


Settings Sync: Support syncing workspace state

Settings Sync: Support syncing workspace state

With the recent addition of Virtual Workspaces, it would be really interesting if its workspace state could be synced too.
I mean, if I have some repository already cloned on any of my machines and I decide to open it virtualy on another machine, VS Code could sync its state, just like it will do when I open it on Codespaces.
Doing so, no matter if I’m opening the repository on virtually or not, VS Code would understand it as being the same.
Thank you


Remove cookie service

In prefs the cookies cause an illegal reflective action to occur which is preventing us from using java16. Cookies are only used by the unitdb  Controller and from what I can tell no relevant cookies to the db are actually stored making it rather useless.

It was supposed to store the unit db settings... Don't ask me if it actually still works :D

Anyways its functionality is not that relevant

Yeah it seems like the type of db is stored in the preferences itself

So it was the setting od @Rackover 's unit db

if I remember correctly you can pass that cookie as a string in a GET parameter, but I may be wrong :)
https://github.com/FAForever/UnitDB/blob/master/index.php#L32 Check it out
Have a nice day ♥

Yeah no idea it is good the cookie service is gone probably


Re-enable the browsertests in Google Chrome on the Windows bot

In PR #14392 we disabled the affected configuration, since the Puppeteer update to version > 13.1.0 caused the browsertests to run extremely slowly and then timeout in Google Chrome on the Windows bot.
It's possible that this is either a bug in Google Chrome, in Puppeteer, or a combination of them both. In any case, we should figure out why this happens and report the bug(s) to the appropriate project(s). (Unless it turns out to be fixable directly in our test-suite, of course.)

It looks like Puppeteer 13.5.2 did improve things, but not up to the point that we can enable it again. The Windows bot did not fully timeout after 60 minutes anymore, but while completing it still took ~40 minutes and Chrome did eventually timeout in the tests themselves.

Let's just WONTFIX this, since I don't think that it's really worth doubling the runtime of the Windows-bot just to improve testing in Google Chrome; please see PR #16490.


Breadcrumb selection in picker is not easy to see

Issue Type: Bug

Choose Visual Studio Light or Visual Studio Light+ as theme
Open a file
Focus breadcrumbs (Ctrl+Shift+;)
Press the down arrow to navigate thru the breadcrumbs
Exactly which item is selected in the picker is too hard to see. (e.g. See "TextEncoder" below.)

(This issue happens in the latest release as well as insiders.)
VS Code version: Code - Insiders 1.28.0-insider (a3f9f2e7240106aeec2f1d4bc03d7876bd42e2e4, 2018-09-07T05:22:14.915Z)
OS version: Linux x64 4.15.0-33-generic

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-5500U CPU @ 2.40GHz (4 x 2388)

GPU Status
2d_canvas: enabledchecker_imaging: disabled_offflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: disabled_softwarerasterization: disabled_softwarevideo_decode: unavailable_softwarevideo_encode: unavailable_softwarewebgl: enabledwebgl2: enabled

Load (avg)
1, 1, 1

Memory (System)
7.71GB (0.58GB free)

Process Argv
/usr/share/code-insiders/code-insiders --unity-launch

Screen Reader
no

VM
0%

Extensions: none

@jrieken Just noticed that yes, explorer tree has same colors. Didn't even realize this was a feature on the explorer tree as I had never "seen" the light blue selection background against the light gray background. I would say this is even a bug on the explorer tree.
For now I have worked around it by setting "breadcrumbPicker.background": "#fff"

I have a PR (https://github.com/Microsoft/vscode/pull/58229) that will slightly brighten the background color of the breadcrumb picker (it will match the picker background color):

Before
After

This list selection color (list.focusBackground) is used in a lot of different places (it's the list widget, I believe) and there are multiple text colors that are used in conjunction with this state that need to pass the color contrast ratio. If we ever update this color we'll need to do a pass of those various states.

@misolori Thanks for looking into this. In your screenshots, I really can't tell which colors have been changed between the Before and After?

If I set ""list.focusBackground": "#add6ff", (the same color as the selection background for text), things look good. As a bonus, the contrast improves everywhere "list.focusBackground" is used, including the explorer tree.

@sambhare thanks for taking a look at this, really appreciate it!
The list.focusBackground is part of the list widget, which that is used in several places (explorer, outline, quick pick, breadcrumb, etc.). The challenge we face is getting all of the colors to work well together while still passing the color contrast ratio for accessibility. Here's a screenshot of the variants:

While I do like that #c7e3ff has great contrast on the breadcrumbs, there's a few areas (updated, filtered) where the colors don't pass:

I'll definitely keep this in mind while I find the right color that works across the various uses of the list widget, appreciate you taking a look!

This is now fixed in master