Getting nested data when using Joins

Currently if i use joins, it returns flat object, whereas returned object should be nested.
knex.select().from("users").leftJoin("user_addresses","users.id","user_addresses.user_id")

returned object is in following format
{
 id: 3,
username: "somename",
email: "someemail",
street: "somestreet",
postcode: "somepostcode"
}

whereas i am expecting
{
  id : 3,
  username : "somename"
  email: "someemail"
 user_addresses:[{
   street: "somestreet",
   postcode: "somepostcode"
}]
}

All mysql query results is flat, you have to do it yourself!
var _ = require('underscore');

knex.select().from("users").leftJoin("user_addresses","users.id","user_addresses.user_id").then(function(data) {
    return _.chain(data).groupBy(function(user) { reuturn user.id; }).map(function(users) {
      var user = _.chain(users).first().pick('id', 'username', 'email');
      var user_addresses = _.map(users, function(u) {
         return { 'street': u.street, 'postcode': u.postcode };
      });
      user.user_addresses = user_addresses;
      return user;
   }).value();
});

Optionally nesting join results would be nice to have

MySQL supports nested tables in query results, however this is not portable to other database drivers as far as I know. Use the .options method to specify driver-specific options per query.
knex.from('users')
  .leftJoin('user_addresses', 'users.id', '=', 'user_addresses.user_id')
  .options({nestTables: true})
  .then(results => {
    // do something with results ...
  });

results is an array of objects with each object having the following shape
{
  users: {
    id: 3,
    username: 'somename',
    email:
  },
  user_addresses: {
    id: 1, // or whatever format your id is in
    user_id: 3,
    street: 'somestreet',
    postcode: 'somepostcode'
  }
}

Disclaimer: the following is a side note barely related to your question.
This is another good reason to name tables using singular form. The table name should represent the entity it is modeling, not the collection of entities that it stores. See this StackOverflow answer for more details.

Boom that's nice

For whom ever stumble across this in the future. Checkout https://github.com/CoursePark/KnexNest. I've took the time to post it here since I don't really understand why this library isn't more popular - it's incredible.

@yohairosen also objection.js and bookshelf allow fetching relations as nested data.

@elhigu These are ORMs. It's a complexity different approach than just mapping raw queries.

full disclosure regarding objection.js would be nice :)

@yohairosen yep fair enough. In those ORMish (not really traditional ORMs since they doesn't try to hide SQL queries behind additional layer of obscurity) things  you need to define relations too according to which nested data is generated, but on the other hand they work also through multiple level of queries so they can also fetch nested arrays of rows effectively like village.houses.tenants which are really inefficient to do with joins due to data amount explosion which joins causes (that is usually the case when I need nested data).
Agreed that for the simple case shown in that README and when that is all you need its 10 times simpler to just use knex nest.
What do you mean by full disclosure? Link to the relevant documentation http://vincit.github.io/objection.js/#eager  maybe?

I honestly did not know a single query with joins can be less efficient than round tripping to the DB. Unless it's being implemented behind the scenes with subqueries?
Do you have more information about this? I'm truly interested...

@yohairosen when using joins with to-many relation, there is lots of data duplication in resulted rows and parsing that data comes really inefficient. Here is some analysis about it https://github.com/Vincit/objection.js/issues/73#issuecomment-224892996

@elhigu Thanks.

Basically it is about the balance between data transfer + parsing in node and with roundtrip times.

So it seems. The discussion in the link doesn't conclude which method is better though. Seems like they have implemented a join query algorithm for you to choose (single query, no roundtrips).
But, as the author mentions later in the discussion, it didn't seem to produce better performance.
That being said, this wasn't a real world test since the DB runs on localhost which, at least in my own apps, is usually not the case. Running the DB on a remote machine would increase the roundtrip times greatly.
Furthermore, parsing the results can be done in workers, which I did not see the author mentions as a solution at all.
Some room for investigation, no doubt.

There was lots of benchmarking/investigation involved I just didn't find the results anywhere with 2min search to the issues. AFAIK there was some numbers released also somewhere.
About using workers, parsing is done by DB drivers, they are quite optimized and some times native code, but afaik they shouldn't use multiple threads because it would not be good for overall performance of the system running hundreds of queries concurrently. Workers won't help overall performance only benefit would be that single query might be processed faster and overall benefit would be a bit worse (in theory) due to extra context switching. Base line is that parsing is driver's responsibility not knex / objection.
If there is 1-1 relation or 1-1-1-1-1-1 relations, join is usually better because they allow filtering whole query with joined attributes and because there are no -> many relations, there wont be overhead in data amount. One has to decide case-by-case which is better / more suitable.
btw. I have locally DB setup behind toxiproxy, which adds network lag simulation when running performance tests of that kind.

yohairosen KnexNest(NestHydrationJS) it isnice however it works only with underscores and is postgres returns data separated by dots it is useless :(

I write a module that does this even it generate types for it
https://github.com/mohaalak/nested-knex

I also wrote a module that does this (simple one using whereIn)
https://github.com/acro5piano/knex-with-relations

For postgres you can you 'json_agg' for one-to-many relations:
knex
  .from('users')
  .leftJoin('user_addresses', 'users.id', '=', 'user_addresses.user_id')
  .select([
    'users.*',
    knext.raw(
      "case when count(user_addresses) = 0 then '[]' else json_agg(user_addresses.*) end",
    ),
  ])
  .groupBy(['users.id', 'user_addresses.user_id'])
  .first();

... which would output:
{
  "id": 3,
  "username": "foo",
  "email": "foo",
  "user_addresses": [
    {
      "id": "?",
      "user_id": 3,
      "street": "foo",
      "postcode": "foo"
    }
  ]
}

if 'user_addresses' was a single entity (one-to-one), you could use following approach:
knex
  .from('users')
  .leftJoin('user_addresses', 'users.id', '=', 'user_addresses.user_id')
  .select(['users.*', knex.raw('(json_agg(user_addresses.*) ->> 0)::json')])
  .groupBy(['users.id', 'user_addresses.user_id'])
  .first();

... which would output:
{
  "id": 3,
  "username": "foo",
  "email": "foo",
  "user_addresses": {
    "id": "?",
    "user_id": 3,
    "street": "foo",
    "postcode": "foo"
  }
}


countersignature: remove TS_VERIFY_CTX_init() call

This is redundant and already handled by TS_VERIFY_CTX_new(). Additionally it causes build failures with LibreSSL >= 3.8.0 which removed this function since it serves no purpose with opaque TS_VERIFY_CTX where it will zero out an already zero'd out ctx or cause a leak.
See: https://github.com/libressl/portable/issues/1006

No Google account, I can't sign the CLA...


[Feature Request] Add Kavita to Wiki for Alternatives

Please add Kavita (https://github.com/Kareadita/Kavita) to the Wiki. I tried cloning repo and wiki, but I was unable to push my feature branch up.
Kavita is an open source alternative which handles Manga, Comics, eBooks.

Oh, I had to disable public edit on the wiki to stop random spammers. I've added Kavita to the list. Thanks for the suggestion, and all the best to the project!


Gmail.check.is_inside_email() seem to not work anymore

There are no items matching .ii.gt .a3s.aXjCH selector anymore on the page

Hey there and thanks for reporting in.
I just did some quick testing in the browser and it seems you are correct. Just relaxing the selector slightly, down to to ".ii.gt .a3s" instead seems to work better now.
Would you be willing to do some testing and see if that works better on your end too? Or if there are unexpected side-effects of such a change?

@josteink Work for me


Fix unicode error python2

Since magic methods __add__ and __radd__ call str on input string, exception is thrown when input string is unicode with non-ascii characters. This PR fixes issue by properly encoding unicode into string on init.

Can this be a thing that is moved forward?

Thank you!


No diagram in document

The 2nd paragraph of the document says "The following diagram demonstrates the relationship between applications ..."
There is no diagram shown in the document.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7ca2544b-ea59-50ea-d831-cd4cc1778326
Version Independent ID: 710c2742-9297-bb9d-d7ec-3a3bd269e8c4
Content: Generate and analyze the CLUSTER.LOG for an availability group - SQL Server Always On
Content Source: docs/database-engine/availability-groups/windows/cluster-log-always-on-availability-groups.md
Product: sql
Technology: high-availability
GitHub Login: @rothja
Microsoft Alias: jroth

@lxnguyen059  Thank you for the feedback.
The documentation has been updated and is pending review.


Wrong warning level in title

Shouldn't this title be listing this as belonging to category warning level 4 (not 3) since the example clearly states that it requires /W4 ?

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 39d1ef07-bd1a-06c5-d02d-98f638ebea09
Version Independent ID: eae492b7-b8bc-e49e-772e-cebbcc9c4e48
Content: Compiler Warning (level 3) C4310
Content Source: docs/error-messages/compiler-warnings/compiler-warning-level-3-c4310.md
Product: visual-cpp
Technology: cpp-diagnostics
GitHub Login: @TylerMSFT
Microsoft Alias: twhitney

Thanks for pointing that out. Fixed.
The fix will be live tomorrow.


xds-k8s driver: increase timeouts for k8s cleanup operations

Added WAIT_MEDIUM_TIMEOUT_SEC: 5 minutes, sleep 10s - used for waiting on deployment start/delete
Increased WAIT_LONG_TIMEOUT_SEC: 10 minutes, sleep 30s - used for waiting on namespace to be deleted

Ref b/178776925

Instead of using fixed namespace and service account names, if we use unique per-run names would we be able to work around the delays in GKE operations?

@sanjaypujare re:

Instead of using fixed namespace and service account names, if we use unique per-run names would we be able to work around the delays in GKE operations?

In short yes, but this is deprioritized at the moment. Tracking: b/177354746

It looks like we still need @ericgribkoff's approval:

At least 1 approving review is required by reviewers with write access.


fix #3600

What have you changed? (mandatory)
Ensure that the cache key is consistent.
What are the type of the changes? (mandatory)
fix #3600
How has this PR been tested? (mandatory)
Check the CI

Signed-off-by:  yangwenmai<EMAIL_ADDRESS>


docs: Improvements to CONTRIBUTING

Added a table of contents
Changed some heading levels to associate the sections with Som
utvikler
Added a section about Github and the workflow
Added a section Som designer with a call for contributions

Click "View" in the top-right in the Files changed tab to see how the document looks :) Inspired by Debugger.html https://github.com/devtools-html/debugger.html/blob/master/CONTRIBUTING.md

@selbekk re English, issue #19 was opened to discuss the language choice for these documents. I think we should come to a conclusion there before adding stuff about language to Contributing :)

Kjempebra!


wxc-searchbar not work in Android

wxc-searchbar does not work in Android because return-key-type is 'next' by default. It could not trigger the return event.
The property  [return-key-type="search"] should be added to the input conponent in wxc-search.

好 ，谢谢反馈，我认为有必要加上


Contributors

This is just a temporary issue, to add contributors . (This will be done in PR's in the future)

@all-contributors please add @Eiromplays  for doc
@all-contributors please add @Eiromplays  for design
@all-contributors please add @Eiromplays  for blog

@all-contributors please add @Eiromplays  for infra
@all-contributors please add @Eiromplays  for maintenance

@all-contributors please add @Eiromplays for doc, design, blog, infra, maintenance, translation, tutorial, security, review


Clean run directory before running tests in run_pyspark_from_build

Implementation for removing contents of the run directory before running tests in run_pyspark_from_build.
The problem, quoted from #6495 is that
"I sometimes run the integration tests across multiple Spark versions. If I run the tests against something like Spark 3.3.0 and then try to run tests against Spark 3.1.x, it will often fail complaining about the metastore version being incompatible. This occurs because remnants from the first run are leaking into the second run via artifacts in the integration tests run directory."
Our solution, as suggested in #6495, is to remove the run directory before running tests in run_pyspark_from_build.

build

build


J2Predictor cannot be used to create passes

~/satellogic/coverage-time-report/venv/lib/python3.7/site-packages/orbit_predictor/predictors/base.py in _get_next_pass(self, location, when_utc, aos_at_dg, limit_date)
    181             raise TypeError("location must be a Location instance")
    182
--> 183         pass_ = PredictedPass(location=location, sate_id=self.sate_id, aos=None, los=None,
    184                               max_elevation_date=None, max_elevation_position=None,
    185                               max_elevation_deg=0, duration_s=0)

AttributeError: 'J2Predictor' object has no attribute 'sate_id'

It probably happens with other predictors, see also https://github.com/satellogic/orbit-predictor/issues/17

(I amend myself: this doesn't have anything to do with #17)


Analytics support: Google tag manager

Implement basic functionality to use Google analytics by Google Tag manager in the application. The instructions can be found here: https://developers.google.com/tag-manager/android/v4/
This is the SDK, preconfigured for this project: https://dl.dropboxusercontent.com/u/3380376/GoogleAnalyticsServicesAndroid.zip
The SDK uses Futurice account (container ID: GTM-5XFNM5) for testing, Futurice employees should have access the web interface. Please make the Container ID configurable.

What will be needed for the implement GTM?


Incorrect warning about Vane v1.8.0 being out-of-date upon server boot & in chat upon login

This may be expected given this is a pre-release, unsure.
[19:17:13 WARN]: [vane-core] A newer version of vane is available online! (current=v1.8.0, new=v1.7.1)
[19:17:13 WARN]: [vane-core] Please update as soon as possible to get the latest features and fixes.
[19:17:13 WARN]: [vane-core] Get the latest release here: https://github.com/oddlama/vane/releases/latest

Yes, this is to be expected as it is a pre-release. Added a note to the pre-release description to point this out. Thanks.


Order polling too fast

Currently the polling for order status happens without any sort of delay. So if the acme client has a really fast connection to the ACME server it will hammer the server with as much HTTP requests as the network allows. If the ACME server has rate limits, the client will probably hit those.
What do you guys think about implementing a delay interval for this polling?
The section of the code I'm talking about is here:
https://github.com/xenolf/lego/blob/d457f70ae0a39ea3e62ddad62caa7c29d33a01e2/acme/client.go#L711-L727

Hmm, good catch, we should probably have a time.Sleep here for... I dunno, maybe 1 second?

Yep. 1 second sounds about right to me.

I think a Ticker can be a solution.

Trying to understand Sleep vs Ticker. Can you explain the advantages @ldez ?
Here is a simple version with sleep in the mean time:
https://github.com/robaman/lego/commit/fbba98bd958e8da2fa22108fa26894ada4671fa7

I think a Ticker creates a more explicit code.
And this allows to define a time out which seems to me better than iterating 1000 times
In addition, one second of sleep can produce a wait of 16 minutes (1000 seconds) or more (related to HTTP calls), which seems too big.
	// ...

	stopTimer := time.NewTimer(10 * time.Second)
	defer stopTimer.Stop()
	retryTick := time.NewTicker(200 * time.Millisecond)
	defer retryTick.Stop()

	for {
		select {
		case <-stopTimer.C:
			return nil, errors.New("timeout, operation failed")
		case <-retryTick.C:
			if _, err := getJSON(order.URL, &retOrder); err != nil {
				return nil, err
			}

			done, err := c.checkCertResponse(retOrder, &certRes, bundle)
			if err != nil {
				return nil, err
			}
			if done {
				return &certRes, nil
			}
		}
	}
}

I like it. Creating PR.

Created PR https://github.com/xenolf/lego/pull/581

Closed by  #581


Roadmap link - error 404

How to reproduce: Help -> Roadmap.
Tested on Atom 1.5.3.

cc @thedaniel ew should probably remove this item


Remove check if openjdk-tests/openjdk-tests directory available

The check is to avoid re-git clone openjdk-tests repo if the openjdk-tests directory exists. However it happens the exist repo is not the one run-aqa expected.
Remove it and use openjdk_testRepo specifically to git clone expected repo.
Signed-off-by: Sophia Guo<EMAIL_ADDRESS>

Besides sometimes   GITHUB_WORKSPACE is set with an incorrect

Tested:
https://github.com/sophia-guo/openjdk-tests/runs/1919789165?check_suite_focus=true


Upgrade to Guzzle v6.

Upgrade to Guzzle v6.
Add public getters for Client and Response so custom steps can be created.
Add assertion for checking JSON without supplying the body.

Shouldn't be any BC breaks directly relating to this package, BUT there will be a BC break if people are using Guzzle v4 elsewhere in their project, without a specifically defined dependency on guzzlehttp/guzzle:~4.0
Because of this, should this be a major version change (eg. 2.0)?
Relates to #28

#28

The issue with this is that it makes the extension incompatible with any project using Guzzle 4 or 5, as they cannot be installed at the same time than Guzzle 6.

I agree, which is why I'm suggesting making this a major version release.

@stof I've reverted the config change back to base_url to prevent BC breaks there, and set the base_uri required for the Guzzle Client in the loadClient method.

You should rebase on master BTW, I changed some things in composer.json, hence the conflicts.

After that it the requirements for guzzle should be 4 - 6, and if would be nice to have travis test with all three major versions

The requirements must be 6 only as it is a complete BC break on Guzzle's side.

I think you should make this P.R. only about upgrading guzzle, it would make things clearer.

@stof @greg0ire
Just starting a new project and I'm worried that without this I could come into version conflict issues later down the line. I'm +1 for a 2.0 release with guzzle v6.
But if you want it to support 4-6 you could write an abstraction/facade to detect a which version of guzzle is installed and run the right stuff, or even have 2 different contexts one for guzzle@4-5 and one for guzzle@6.
What needs to happen to get the ball rolling again on this? @nspyke can you rebase?

closing in favor of #35 which keeps support for Guzzle 4 and 5 too, making it much friendlier to projects using them (and avoiding to have a new major version just for that, which would then force to maintain 2 versions of the extension in parallel)


dependency on your assertion module

Hi this is so far a great module, did the job and seems like a robust strategy for nested maps (I did not check how it will handle lists). Thanks for sharing this. One issue:
There is a dependency on your assertion module which prevents use of your deepmerge module from the terraform registry because it has a hardcoded source path of "../assert" to assert the merge worked. The assertion is a good thing in itself, but you need to fix this. There are a few of ways I can think of, don't know which one you would prefer so I can't submit a PR:

use the very popular "assert" module that is available in the terraform registry: https://registry.terraform.io/modules/kchugalinskiy/assert/null/latest
publish your assertion module and reference it via registry instead of relative local path
include your assertion module as a submodule, and change ../assert to ./assert

Meanwhile, my workaround was that I downloaded a gzip of your two modules (deepmerge and assertion), unzipped them in a subfolder of my own module, and renamed the assertion folder to "assert". My code looks like this:
my-tf-module
  inviction-lab
     assert
     deepmerge
     README.md // I point to your github
  main.tf
  ...

Ah, sorry about that! The assertion module was already published in the registry, but I hadn't updated this code yet. The new release (v0.1.1) should be good.


Add JVS INGT10 GQS60EP T10A+OV9750

In my opinion, these actions are unnecessary We need to make sure that this driver is compiled from source directly into Firmware along with the rest.
devices/t10_lite_jvs-ingt10-gqs60ep/general/package/ingenic-opensdk/0001-add-ov9750.patch
devices/t10_lite_jvs-ingt10-gqs60ep/general/package/ingenic-osdrv-t20/ingenic-osdrv-t20.mk

At the moment there is no other solution, you need to either make a patch adding a sensor to openingenic or add an already compiled driver to the assembly.

Okay, then this is the proposal.
We accept this patch, but at the same time open a ticket in the openingenic repo about the need to create a patch for this driver.
Once created and verified in openingenic, I think we can remove the files from the builder repo.
I mean that you need to try to do as much as possible in other repositories so that the builder repo remains minimalistic for each device.
Thanks for understanding.

You need to enable compilation of this sensor in the openingenic repository and, after checking, remove the files from the device to the ingenic repo. We need to strive for universality, so that as few change files as possible are in the Builder repo.
Only a package configuration file, an unnecessary removal file, a settings file. Three files are the minimalism you need to strive for


Missing error check of RAND_bytes() calls in apps/speed.c

Shall we check the return value of RAND_bytes() in apps/speed.c? Such as,
speed.c : 881-890
static int RAND_bytes_loop(void *args)
{
    loopargs_t *tempargs = *(loopargs_t **) args;
    unsigned char *buf = tempargs->buf;
    int count;

    for (count = 0; COND(c[D_RAND][testnum]); count++)
        RAND_bytes(buf, lengths[testnum]);
    return count;
}

We find the return value of this call been checked in other files in apps folder.
Such as,
apps/s_cb.c
708:         if (RAND_bytes(cookie_secret, COOKIE_SECRET_LENGTH) <= 0) {
709              BIO_printf(bio_err, "error setting random cookie secret\n");

apps/rand.c
107:         r = RAND_bytes(buf, chunk);
108          if (r <= 0)
109              goto end;

In the manual doc, RAND_bytes() puts num cryptographically strong pseudo-random bytes into buf and the return value 0 on failure. This is the doc link
This problem exists in many versions:
OpenSSL_1_1_0-stable/apps/speed.c,
OpenSSL_1_1_1-pre8/apps/speed.c,
OpenSSL_1_1_1-pre4/apps/speed.c, et al.
Zuxing Gu, Jiecheng Wu
Tsinghua University, Tsmart-IMChecker Group
<EMAIL_ADDRESS><EMAIL_ADDRESS>

Can you open pull requests for all those issues you've filed?