Reduce Memory Consumption Used to Invoke a Single Action

Reduces number of call-by-value references to action code.

@rabbah, @markusthoemmes, can you have a look. Haven't looked through sequences yet.

Have you measured the impact of this?
Note that "call-by-value" for instances means: passing a reference, which is virtually free in terms of heap consumption.

While it's true that, in theory, not holding a reference to the action code could make it possible to release the heap consumed by the action code (our cache though holds a weak-reference as well), in this case we would speak about milliseconds of delay pre-change vs. Post-change.
Thats is: If we're leaking a reference to a closure that lives for longer (we used to do that with activations), that would indeed be a real problem.

@starpit memory profiled a lot of the controller. Nick - thoughts?

i suggest that, before we proceed (and, indeed, for any memory-related change, if not perhaps even for any controller change going forwards) that we should collect:
at the very least: max heap occupancy for a test which invokes a longish sequence N-way, in parallel. compare this value to that of the current master.
very important also: collect a class histogram after completion of the test, say within 30 seconds (very important: less than 5 minutes after test completion), and ensure that the number of WhiskActivation structures is very small; if N threads each execute a sequence of length M, and to his L times, then the number of WhiskActivation structures alive should be close to N-ish, some small multiple of it at worst.
ideally, we also want the instantaneous number of JSON structures to be close to N (if that is max concurrency for the load test), and the number should decrease to zero within 10 seconds of test completion.
beyond this, we want the number of JSON structures in heap to be constant, independent of the number of concurrently live actors. this will take quite a bit more work, but is probably the single biggest offender.
beyond WhiskActivation and JSON structures, we also want to establish the property that the number of code/blob strings in heap is zero. we should get to this point, and write tests that establish the verity of this condition, by using class histogram snapshots taken during such a mild load test run (i.e. we don't need to do a heavy duty load test in order to identify these problems, and to verify that they remain fixed). but it does require some amount of load, i.e. these are just unit tests.


'Load balancing' should just say 'load' or 'server load'

In ElkArte 1.1.0 Beta1, the web page at Admin > Main > Support & Credits, Support Information lists the following:

Load Balancing: 0.38

I would think this value actually represents the system load average for the past 1 minute, as returned by the 'uptime' and 'w' commands on Unix-like systems. If so, this is unrelated to 'Load balancing' which is an very different concept, and thus a misnomer.

Correct, its the 1min output from the built in PHP sys_getloadavg function.  Looks like I was a bit lazy when I resued the txt sting :innocent:

The term Load Balancing is also used at Administration > Configuration > Server settings > Load Balancing. This setting is about how to react to situations of elevated server load. This is still not 'load balancing', but you may wish to find a more specific and more descriptive term than just 'load' for this setting. Possibly 'high load' or 'managing load' might work. Or you could come up with something better. ;-)

Server Load could do?

I guess it works in both places, although it is used in different contexts.

@tomreyn have a look at the changes made by @live627 in #2678 and let us know if it improves the situation. :)

This looks good (and much better than before) to me, thanks to both of you!
I'm unsure as to whether or not the change in line 707 is logically correct (negation removed), but I'm sure live627 can better tell than I.

He removed the negation and inverted the statements, so it's correct. :D


Feature Request/Question: Given a chart file can I list all the images that will be needed?

My current use case is this: my team and I have been developing some charts for a client and we've found out recently that our requirements are 100% bare-metal and isolated from the outside world.
This isn't a blocking issue, because we can deploy to any arbitrary cluster and run:
kubectl get pods --all-namespaces -o jsonpath="{..image}" | tr " " "\n" | sort --unique

and we now know what images we need to docker save and then later docker load.
To create a more performant build process for our final deploy, is there a way we can have helm spit out all the images that are required for a chart (including subcharts) so we can automate this without having to deploy to a cluster first?

I'm sure this would be a useful plugin if someone were to parse the output of helm template and show what images are used in the chart. This functionality is not available as Helm, though.


board安装包adminserver中git仓库应修改为默认为gogit

In GitLab by @cuiaq123 on Aug 6, 2020, 03:08

In GitLab by @JackZxj on Aug 7, 2020, 02:05
assigned to @wknet123 and unassigned @JackZxj

In GitLab by @JackZxj on Aug 7, 2020, 02:06
默认值是从后台的config读出来的。

如果需要修改为gogs的话请 @wknet123 更改默认值即可。

In GitLab by @cuiaq123 on Aug 13, 2020, 06:22
closed


Unable to run tests after upgrading to EG2.0 with cocoapods. Library not loaded: @rpath/AppFramework.framework/AppFramework

Updated to EG2.0 with CocoaPods by following this video https://www.youtube.com/watch?v=KQaA11qVQn0
When run the tests, build succeed but test failed, error logs:
The bundle “EarlGreyUITests” couldn’t be loaded because it is damaged or missing necessary resources.
Try reinstalling the bundle.
2020-09-03 12:05:39.707775-0700 EarlGreyUITests-Runner[8009:63716] (dlopen_preflight(/Users/xx/Library/Developer/Xcode/DerivedData/xx-dhjlwvppmpfilpegrixtzigccydw/Build/Products/Debug-iphonesimulator/EarlGreyUITests-Runner.app/PlugIns/EarlGreyUITests.xctest/EarlGreyUITests):
Library not loaded: @rpath/AppFramework.framework/AppFramework

Referenced from: /Users/xx/Library/Developer/Xcode/DerivedData/xx-dhjlwvppmpfilpegrixtzigccydw/Build/Products/Debug-iphonesimulator/EarlGreyUITests-Runner.app/PlugIns/EarlGreyUITests.xctest/EarlGreyUITests

Reason: image not found)

Hello, now I have the same problem：dyld[6772]: Library not loaded: @rpath/AppFramework.framework/AppFramework
Referenced from:  /private/var/containers/Bundle/Application/72BC7E92-E0CA-466E-AC3F-E7C198FF8BE4/TestCodeUp.app/TestCodeUp
Reason: tried: '/private/var/containers/Bundle/Application/72BC7E92-E0CA-466E-AC3F-E7C198FF8BE4/TestCodeUp.app/Frameworks/AppFramework.framework/AppFramework' (mach-o file (/private/var/containers/Bundle/Application/72BC7E92-E0CA-466E-AC3F-E7C198FF8BE4/TestCodeUp.app/Frameworks/AppFramework.framework/AppFramework), but incompatible platform (have 'iOS-sim', need 'iOS'))，Do you have a solution？thanks


feat: production ready

I know there is a lot here but that's what I needed to make this production ready.

fix some typings errors
fix test not closing
add basic cors support
fix onAborted issue.
upgrade to latest uWebSockets.js
review the whole WS part to make this optimized, and compute context while upgrade
use the common parts that were merged from my pr to trpc (https://github.com/trpc/trpc/pull/4106)
I let the jest/vitest things I don't know what you want to do with that, your test:coverage command does not work

Hello,
I have implemented most of your changes.
But I have left out the following:

cors support: it can be implemented easily outside of the trpc handler, and can be fully customized by end user
compute context while upgrade: I want to keep internals of this adapters similar to ws implementation. In your implementation the client will never know that createContext failed, so it will attempt to reconnect to server forever

Oh great I will test your changes very soon.
I can't look at the code right now but outside cors doesnt seem to work because we need it inside the handler too, not only add an .options handler.
For the WS part, my code send a closeUpgrade, so yes, the client know, this code is already in production and that's all the changes needed to make it optimized and safe. I will compare with your version, and put some benchmarks here. Hope yours is safe and optimized enough then

Alright, thank you for taking your time to test this.
In the unit tests I have tested your closeUpgrade which returns 403 instead of 101. The client in this case keeps trying to reconnect to the server, as it thinks that there is an issue with the protocol. Trpc have not finalized subscription api, so its rough around the edges.
Also, I haven't tested this, but maybe you need to define OPTIONS route before trpc, so that it takes higher priority. This is exactly how it works in your handler, so why would it not work outside of it?

didn't have time to tackle this yet, but yeah you probably right for the upgrade part but might be easy to fix. But do you try to stress test your way?
For cors, you have to send the Access-Control-Allow-Origin during the post request too so we have to handle this in this package

Yes I agree, having a simple to use plugin is nice. When learning webdev I suffered with this cors thing too much.
Its possible for trpc server to add headers on each request, as can be seen here. This is implemented internally in trpc, so I assume it works in this library. Try it out!

Closing this as all functional parts were integrated. I have also added CORS to the readme.


hiero: precollect instances failing when audio selected

Description
If user select clip and related audio clips before publishing the precolect_instances plugin will fail.

/cib


Fix #100: Add pylint in workflow

Fix #100
Adds pylint in the workflow to lint all python files in this repo.

Hi @U8NWXD , I have addressed all your comments and also added the pylint in the wiki, Please let me if it is in the right place or needs any other changes. Thanks,

@U8NWXD I know you are busy but just making sure you didn't missed it

Sorry for the delay, and thanks for the reminder!

Hey @U8NWXD PTAL. Thanks,


spike: Experiment Candidate for Leader Elections ♠

For reference:
Starknet Decentralized Protocol I - Introduction
Starknet Decentralized Protocol II - Candidate for Leader Elections
Starknet Decentralized Protocol III - Consensus
Starknet Decentralized Protocol IV - Proofs in the Protocol
Starknet Decentralized Protocol V - Checkpoints for Fast Finality
Starknet Decentralized Protocol VI - The Buffer Problem
Starknet Decentralized Protocol VII - Chained Proof Protocols & Braiding

Madara Consensus Spike:
TASKS

Sync on outer node storage & "mapping-sync" on node init(extend for consensus data ref moonbeam 'relay_chain_state_proof')
Create "bookie" solidity contract/and $fSTARK(contact Ilia to see what's been done)
Create "bookie" block author pallet
Create "quorum" finalization pallet
Benchmark/Integration tests
Evaluate usage of HotStuff pallet when available

Architecture
Substrate Consensus done in 2 phases:

Block Authoring process nodes use to create new blocks
Block Finalization process used to handle forks and choose canonical chain

Block Authoring:
Aura and Babe require a known set of validators or "authorities" as block producers, time is divided into discrete slots and during each slot authority can propose block.
Bookie(StarkNet Proposal) - epoch based stake weighted scheduling
- epoch(5m)/slot(15s) public lottery based on L1 timestamps
- leader_election(epoch randomness, stake distribution) - can be calculated offchain
- state update of the L1 core contract ensures via proof that L2 block authorship follows schedule
alts/refs
Aura - Authority Based Round Robin Scheudling
- rotating list of authorities takee turns creating blocks
- majority of online authorities are assumed to be honest
- used in combo w/ GRNADPA where Aura = block production and short term finality and GRANDPA deterministic finality provided by GRANDPA
Babe - Blind assignment of blockchain extention slot-based scheduling
- authorities win slots based on a verifiable random function as opposed to round-robin
- winning authority can select a chain and submit new block
Block Finalization:
Substrate exposes the fork choice rule implementation via the "SelectChain" trait and you can use custom or GRANDPA. Substrate exposes "LongestChain" for chain selection rule.
Deterministic finality can be reached with a set of fixed authorities casting "finality" votes. When enough(2/3) votes have been cast block is deemed final. Blocks that have been finalized can't be reverted w/o external coordination such as hard fork. If your blockchain has a finality mechanism, must be modified wthe fork choise rule to consider the finality vote.
Quorum(StarkNet Proposal) - dual ledger finality via quorum certs
- Periodic L1 checkpointing of state update
- Quorum checkpoints supported by majority of stake
- Conflicting checkpoints can't exist
alts/refs
GRANDPA - deterministic finality mechanism for blockchains
- longest chain rule = best chain is the longest chain
GHOST - greediest heaviest observed subtree
- the correct fork is the one that has the most blocks built on it
HotStuff - 3 phase view-change w/ quorum certs message exchange
- Random/Fair/Rotating primary(no need for seperate block authoring pallet, can use randomness)
- Quorum Certificates consist of unique node and view number
- (prepare)Leader extends branch with the highest QC view and broadcasts replica accepts/responds w/ portion of threshold signature -> (pre-commit) Leader combines in a "prepare QC" and broadcasts prepare message and replicas vote on pre-commit -> (commit) Leader combines votes in "pre-commit QC" and broadcasts commit message, replica votes on commit and locks "pre-commit QC" ->  (decide) leader combines in "commit QC" broadcast decision, replica execute command, increase view number,start new view
- Substrate(madara) impl https://github.com/keep-starknet-strange/madara/issues/180

project repo for PoC solidity contracts:
https://github.com/keep-starknet-strange/zaun


Error while using pretrained model

Hi
I'm trying to feed a point cloud that has been generated with open3d to a pretrained PCN but I get this error :
'open3d.cpu.pybind.geometry.PointCloud' object has no attribute 'shape'
Can you please help me? Thanks a lot.

hi, the input should be torch.Tensor, try to extract the point cloud out as coordinates by use '.points'.
ref: http://www.open3d.org/docs/0.14.1/python_api/open3d.geometry.PointCloud.html

Hello Again. Thanks a lot for your answer.
I did what you said. First I get the points then convert them into numpy array and then into tensor.
Now I'm getting a new error:
... ,  in forward
    bs , n , _ = xyz.shape
ValueError: not enough values to unpack (expected 3, got 2)

the input size is (number of points , 3) = (4472 , 3) .
Do you have any idea how to solve this error?
Thanks again :)

Hi @Ali-Rashidi ，
Sorry for the late reply. Have you solve your problem? When you use your own point cloud data, you should:

Downsample it to 2048 point (opinional)
Unsqueeze the Tensor to 3 demension (B N 3). For example, (4472, 3) -> (1, 4472, 3) by using xyz.unsqueeze(0)

Feel free to reopen the issue.
Best regards

Hello
Tnx for your reply
I solved the problem.
Tnx a lot


Add Sign-Up Page and link with home section

Describe the bug
The sign-up page link is missing or not properly added to the website’s home section. Users are unable to access the sign-up page from the homepage.
To Reproduce
Steps to reproduce the behavior:

Go to the website’s homepage.
Look for the "Sign Up" link/button in the header or navigation bar.
Click on the "Sign Up" link.
Either the link is missing, or clicking on the link does not redirect to the sign-up page.

Expected behavior
The "Sign Up" link should be visible and easily accessible from the homepage, and clicking on it should take the user to the sign-up form.
Additional context
Add any other context about the problem here.

Kindly refork and resync

Why

As the latest repo are merged
So there will be no merged pr issue if you have resolved

Ok

Have you done

Yes
Please add level 3 sir as I have to create new two pages for the same

You are only adding but it's validation and integration through backend will be level3

PR already made for same issue by someone else


Please fix apollo.sh lint


VLAZ: Watch teamsli.de

VLAZ requests the watch of the watch_keyword teamsli\.de. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

teamsli\.de has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by cigien in Charcoal HQ


Gradle Script failing to compile

I apologize as, based on previous commentary, this seems like a recurring theme.
I followed the instructions from the README regarding configuration.  However, gradle is still failing to compile, providing the 'Failed to resolve' message.
I don't know if this matters, but I am currently using the following versions for the application in gradle and I am trying to configure with your most recent build from the readme.
compile 'com.android.support:support-v4:22.2.0'
compile 'com.android.support:appcompat-v7:22.2.0'
Any assistance regarding this venture is much appreciated.  Thanks for your time.

You'll probably need to use version 23.1.0 of Google's libraries, and use Build Tools 23.0.1.
Otherwise there's not much I can do to help unfortunately.

It continues to fail even with the most modern tools and libraries.  May I ask why this appears to be a recurring issue with the library?

I've never been able to figure it out.
Every time I publish a release, I immediately reference it in all of my personal apps and employer's apps, and it works fine.
JitPack.io also reports that the latest version built successfully their servers: https://jitpack.io/#afollestad/material-dialogs

I'd have to guess that most of the people who have these errors aren't setting up their Gradle files correctly.

The reason I ask is because I am inheriting a project that is utilizing this library.
Additionally, just as a reference, a blank android project will cause gradle to fail with the instructions provided in the README.  Are there any other configurations required besides the 2 steps listed?

@mmkgitpush456 as long as it looks like this in your module build.gradle file (not your project's build.gradle file), there shouldn't be anything wrong:

Thanks, now it works.  The project is using an older version of the API.  The newest release seems to have some collisions with the styling, but that can saved for another day.
The most important thing was just getting this guy to compile correctly.
I greatly appreciate your assistance on this matter.

Great, glad it works!

If you want to force using your own support library version, something like following works as well:
compile('com.afollestad.material-dialogs:core:<IP_ADDRESS>@aar') {
    transitive = true;
    exclude group: 'com.android.support', module: 'appcompat-v7';
}
compile('com.afollestad.material-dialogs:commons:<IP_ADDRESS>@aar') {
    transitive = true;
    exclude group: 'com.android.support', module: 'appcompat-v7';
}

and include whatever suppor library version you need and want...

The library is now on jCenter, as an alternative to JitPack.io. jCenter is Android Studio's default Maven repository.
See the README for updated dependency info: https://github.com/afollestad/material-dialogs#gradle-dependency


djangoUrl relative vs absolute url

The 'djangoUrl' returned from the server is relative and not absolute.  This has caused an issue getting form submissions to work correctly in a development environment as it would attempt to contact the frontend instead of the backend.
I have changed the following to return an absolute url.  Not entirely sure if this is an issue or I have overlooked a simple fix.
Line 326 in 'rest.py'
Original
ret['djangoUrl'] = base_path + self._get_url_by_form_id(form_name)
Changed
ret['djangoUrl']` = self.request.build_absolute_uri(base_path + self._get_url_by_form_id(form_name))

@sssolid : thanks for the issue. In my development scenario I do not have it as I'm using angular cli with proxying on, that is:
ng serve --proxy-config proxy.conf.json
... running on <IP_ADDRESS>:4200
cat proxy.conf.json
{
    "/api": {
        "target": "http://<IP_ADDRESS>:8010",
        "secure": false,
        "logLevel": "debug"
    }
}

which for example proxies http://<IP_ADDRESS>:4200/api/blah to http://<IP_ADDRESS>:8010/api/blah.
In production the setup is similar with nginx handling angular files and proxying django.
During the weekend I'll look if this change is ok with our deployment scenario
(I'm not sure it the development proxy passes the port&path correctly) and will
add absolute/relative urls as an option to django settings.
Thanks again, ms.

Fixed in 6.1.2, I will have to write some tests for that :)


Upgrade log4j to 2.17.1

Change-Id: I604ece14cbfebde86f8c33752aa1e13b7061433f
Description
Brief description of what this PR is about

If this change is a backward incompatible change, why must this change be made?
Interesting edge cases to note here

Codecov Report

Merging #1444 (7f96e18) into master (bb5073f) will decrease coverage by 1.37%.
The diff coverage is 47.84%.

@@             Coverage Diff              @@
##             master    #1444      +/-   ##
============================================
- Coverage     72.08%   70.71%   -1.38%
- Complexity     5126     5299     +173
============================================
  Files           473      493      +20
  Lines         21970    23217    +1247
  Branches       2351     2532     +181
============================================
+ Hits          15838    16417     +579
- Misses         4925     5521     +596
- Partials       1207     1279      +72

Impacted Files
Coverage Δ

api/src/main/java/ai/djl/modality/cv/Image.java
69.23% <ø> (-4.11%)
:arrow_down:

...i/djl/modality/cv/translator/BigGANTranslator.java
21.42% <ø> (-5.24%)
:arrow_down:

...odality/cv/translator/BigGANTranslatorFactory.java
33.33% <0.00%> (+8.33%)
:arrow_up:

...nslator/InstanceSegmentationTranslatorFactory.java
14.28% <0.00%> (-3.90%)
:arrow_down:

.../modality/cv/translator/YoloTranslatorFactory.java
8.33% <0.00%> (-1.67%)
:arrow_down:

...i/djl/modality/cv/translator/YoloV5Translator.java
5.69% <0.00%> (ø)

...odality/cv/translator/YoloV5TranslatorFactory.java
8.33% <0.00%> (-1.67%)
:arrow_down:

...modality/nlp/embedding/TrainableWordEmbedding.java
45.00% <0.00%> (-3.34%)
:arrow_down:

...pi/src/main/java/ai/djl/ndarray/BytesSupplier.java
54.54% <0.00%> (-12.13%)
:arrow_down:

...i/src/main/java/ai/djl/ndarray/NDArrayAdapter.java
15.23% <0.00%> (-0.83%)
:arrow_down:

... and 158 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c41b6a0...7f96e18. Read the comment docs.


IEMOCAP recipe - missing lengths in call to embedding module ?

Hello Speechbrain team,
While trying to reproduce the results mentioned in https://github.com/speechbrain/speechbrain/tree/develop/recipes/IEMOCAP I figured out that the accuracy on the test set varies w.r.t the batch size in the test set dataloader, sometimes reducing the accuracy by up to 7% when batch_size = 1.
Further investigations in the code made me think that the 'lengths' parameter might be missing in the call to the embedding module (train.py line 37).
I tried to change it to:
embeddings = self.modules.embedding_model(feats, lengths=lens)
and ran the train and test stages again.  Then the variations are less important but are still present : around 1% difference from one test to another (train stage done only once).
So I have 2 questions:

Do you think the modification hereabove is correct?
What could explain the remaining fluctuations with respect to the batch size of the test set?

Thank you in advance

I think the correct way is to pass the lenghts  parameter to the embedding model (this way zero padded elements are removed from the statistical pooling operation). @aheba, can we do it in your ongoing PR?

sorry，I didn't understand how to change this in the train_with_wav2vec2.yaml config file :
The train logger writes training statistics to a file, as well as stdout.
train_logger: !new:speechbrain.utils.train_logger.FileTrainLogger
save_file: !ref <train_log>
seed: 1993
__set_seed: !apply:torch.manual_seed [!ref ]
Should I change this things after the "!"  ?
thx.


apt: Pull ansible from Stretch

This should help making sure all admins have access to an Ansible 2 install, for hashbang/shell-etc#14.

23:38:59           ~Ryan │ KellerFuchs: #79 looks good


Implement Pathname#join

Rust seems to have it's own path joining methods. http://rustbyexample.com/std_misc/path.html  This "may" do exactly what we want.

[ ] Write Rust implementation of Pathname#join to FasterPath.join
[ ] Write refinements and monkey-patches for Pathname#join and map implementation to Pathname.new(result)
[ ] Write tests from existing sources proving FasterPath.join operates the same as Pathname#join
[ ] Write benchmarks in test/benches and compare the performance.

@danielpclark I'd like to attempt adding FaterPath.join

Have at it :-)


Issue trying to get gcloud / google dns working with Lego

Hi,
I am trying to test Lego with Google DNS via the gcloud DNS provider.  My issue is, i can not work out what vars need set to get Lego to authnticate to my Google account :(
I have the gcloud CLI utility installed, and have authenticated to my google account.  but i dont know what i need to set for Lego to authticate, anyone able to give me some pointers and examples?
thanks
matt
P.S. this is my first time using gcloud as well as lego :(

Hey there!
The DNS provider for google cloud DNS uses the credentials provided by the CLI utility so you are fine on that part. What lego needs is only an ENV variable GCE_PROJECT set to your project.

I assume you solved your problem?
Feel free to re-open if that's not the case.


State of django integration for Strawberry

As a novice in graphql it seems quite confusing about the state of graphql and its python and django integration - you really feel more in a npm environment that in a pip environment :) I came across strawberry because I was told that the most starred graphql library for django is dead and stumble across strawberry which neatly integrates some cool new and useful Python features like it type hinting.
My proposal would be to state the following motivations in the README.md to better understand the state of strawberry-django which is essential if you consider using it for a project

why strawberry vs. graphene-django

why is there strawberry.django, strawberry_django - this repo, strawberry_django_plus (I know there is a PR  https://github.com/strawberry-graphql/strawberry-graphql-django/issues/139) and some extensions for ??
What are their dependency hierarchies as they somehow depend on each other but at the same time also have no fixed dependencies, see https://github.com/strawberry-graphql/strawberry-graphql-django/issues/182#issuecomment-1260642217 - IMO it is best to have an upper boundary within a minor version (so that the API is stable) instead of having a lower boundary, e.g. the following line will break in production and dev easily if you don't freeze your complete dependency tree and requires you to always use the newest version of everything https://github.com/strawberry-graphql/strawberry-graphql-django/blob/b615ed58a75943aacb0f41faa77d3a5bfedf2ea6/pyproject.toml#L29
I think some confusion could also be resolved if the documentation would use sphinx (#207 ) as this also allows to link the documentation across multiple projects

why does this django integration deviate from common django design principles and more follows the strawberry approach?  DRF mimics the Django Form API, making it easier to adapt and understand/follow what is happening behind the abstraction. IMO an inheritance pattern is more easy to understand than a decorator based approach - this is maybe a matter of preference but Django uses decorators only to encapsulate (e.g. @login_required, but which can also be used as a Mixin), rarely to generate code, e.g. the current approach
@strawberry.django.type(models.Fruit)
class Fruit:
    id: auto
    name: auto

would look more like this in a traditional Django based approach (e.g. Admin, Form, ClassBasedViews)
class Fruit(strawberry.django.Type):
  id: auto
  name: auto

  class Meta:
    model = models.Fruit

The class inheritance make it easier to understand what is happening from within the IDE as well - therefore I would be really interested what the motivation behind the currently used design principle is.

Also some crucial use cases like authentication are stated as done on the README.md but is not documented and is also asked on SO - some patterns for permissions would also be nice :)
I also look forward to contribute to this repo as I decided to use it in a project :)

Hi @capital-G ,
There's still a lot of improvements to be made to this lib indeed, specially regarding documentation. I'll answer some of your questions here:

why is there strawberry.django, strawberry_django - this repo, strawberry_django_plus (I know there is a PR https://github.com/strawberry-graphql/strawberry-graphql-django/issues/139) and some extensions for ??

The main strawberry repository is a monorepo containing all of the integrations with other frameworks, and the idea is to have this integration there as well once it is mature enough.
Also, since you mentioned strawberry-django-plus, it is an extension to this lib that I wrote at the end of last year to add some functionality that I needed for some projects that I was migrating from graphene. It end ended up having a lot of very useful (some might say "must have") features that the community liked (i.e. the query optimizer). I later was invited to maintain this repo and decided to merge those features here, which I unfortunately didn't have time yet to do yet =p.

why does this django integration deviate from common django design principles and more follows the strawberry approach? DRF mimics the Django Form API, making it easier to adapt and understand/follow what is happening behind the abstraction. IMO an inheritance pattern is more easy to understand than a decorator based approach - this is maybe a matter of preference but Django uses decorators only to encapsulate (e.g. @login_required, but which can also be used as a Mixin), rarely to generate code, e.g. the current approach

This project and strawberry itself follows the patterns defined by the dataclasses lib, in which a decorated class gets transformed to become "a data class". I understand when you say that it deviates from the django design principles, but following it would also deviate from the strawberry design principles.
Also note that when you are writing a graphql api you are not only exposing types that gets "automatically generated" by the ORM extension, but you are also writing other pure strawberry types and also adding non ORM fields to your type. If this lib followed DRF conventions you would end up with a lot of mixed styles in your module that don't talk to each other. Not to mention that the DRF style doesn't support data class transforms.

I also look forward to contribute to this repo as I decided to use it in a project :)

All contributions are very welcomed! :) As I mentioned earlier, there's still a lot of work to be done here. There are a lot of features to be ported from strawberry-django-plus, there's some missing features and improvements to be made, and a lot of documentation to be written.
Hit us on our discord channel in case you want to discuss anything, are having any issues. Feel free to @ me there as well :)

Yeah, there's a lot of fragmentation and still a lot missing. Having got one django app in production and working nicely with graphene, I've begun working with strawberry after reading the exact same post as @capital-G and found that reality is that there's a hell of a long way to go before we hit parity with the django graphene offerings.
To be clear, this isn't a dig at you @bellini666 since you seem to be overwhelmingly delivering for an ever-bigger set of user demands.
Is the constraint for you and the rest of the core strawberry team about time, or about money?

If the latter, I can't afford to become a recurring sponsor but potentially can sponsor some specific issues if you'd like to talk more.
If the former, perhaps I or my team at Octue do a few things but we're also pretty resource constrained, and honestly we're still learning about strawberry so it might not be so effective.

I disagree with the Graphene vs Strawberry parity remark @thclark.  Recently doing a large project with strawberry, I've mainly found it to be a steep learning curve with some rough edges that need smoothing.  But feature wise, Strawberry out-classes graphene for our projects any day of the week.
Was it more difficult to get started then graphene? Yes.
Did I end up doing some changes myself to fit my use-case? Yes.
Would it have been faster with graphene?  Maybe - but that library is pre-async era and has plenty of challenges and missing features itself.
Let's compare Strawberry to graphene with a few quick real-world examples:

Strawberry
Graphene

Async / Sync
Async first
Sync First

Subscriptions
out of the box
community

CUD mutations
out of the box
create yourself

File upload.
out of the box
community.

You can see that when it comes to foundations, Strawberry is the winner.  The rest, is us as a community improving and chipping in one commit at a time.
I'm on the strawberry side, hands down.  One of the main challenges I see in the learning-curve of Strawberry-django vs graphene.  Why?  The code-style challenges how we've come to expect a Django application to work. That for me created some head scratching, and some frustration along the way.  A improved set of documentation that's writing from a new user perspective would have cut down my onboarding time in half.
Strawberry is a diamond in the rough.
@bellini666 in between the lines I understand that Strawberry is on top of your daily responsibilities.  Which I would imagine makes it very difficult giving the project constant attention.
During my time working on the library, I would have happily paid someone with the experience to get things moving along.  The sponsorship opportunities didn't allow for this philosophy, so we ended up investing the time ourselves.
Personally I would happily setup a commercial support for Strawberry, which will ultimately benefit the library itself, everyone using it and add to guaranteeing long-term succes.
I believe this would be the right way forward to increase traction.  Drop me a msg if you're open to discussing this.

Hey @thclark and @sdobbelaere ,
First of all, as a former Graphene user myself, I want to agree with @sdobbelaere regarding strawberry being superior! Of course my opinion should be taken with a grain of salt as I'm biased towards it :)
One thing for sure that Strawberry Django needs is more docs and tutorials! I really want to start creating some video tutorials to post on youtube for it, but I'm lackin time lately. An introduction could be this presentation I made at last year's Djangocon US: https://www.youtube.com/watch?v=TP8MC2W3eIw

Regarding my dedication to this repo: At my previous job I was using strawberry extensively to build a lot of projects, but I recently moved to another job where I'm unfortunately working with REST now T_T
I'm still committed to maintain this library and will never abandon it no matter what, but while before it was part of my daily work to identify issues and improvement opportunities as one of the users of this lib, now this has become a side-project and I can only dedicate my spare time to it.
During that spare time I'll usually focus on reviewing PRs from the community, replying to issues and also trying to tackle some issues that I think are important, but for sure it is not moving as faster as it was before... Having said that, I'm open to have some issues/features sponsored to give them a high priority in that spare time :). Feel free to send me an email (look at my profile) or send me a pm on discord (my username is bellini)
Also, regarding setting up a commercial support for Strawberry and ensure its longevity, it is possible to support it at our open collective page: https://opencollective.com/strawberry-graphql

Thanks @bellini666 and @sdobbelaere these are helpful comments.
On my parity comment, our difference is an issue of perspective - but the main thing is it's worth reiterating that I work in OSS myself, so I do really understand the struggle; I'm tremendously grateful of your efforts, as they're what brought us here!
Let's put some polish on the diamond, to follow @sdobbelaere's simile.
It sounds like @bellini666 you're struggling a bit for time but in the first instance I'll drop you an email and let's discuss properly how I can support you. I'm a good documentation person so perhaps we can work out something where I sponsor to prioritise some features, then contribute some days of work on the documentation myself once I see how it all works.
In the meantime I'll make an issue around subscriptions, which is one of the areas I'm interested in bringing strawberry-django along.


Add dodgewindows to hiding for plasma 6

Plasma 6 includes a dodgewindows option for panels. I'm not sure if backwards compatibility is a concern here. However, this seemingly isn't a concern with the floating option.

Nice spot, thanks. I'll merge right away!


The future of this library: Improve bootstrapping, initialization, and remove common developer errors

In a meeting between me, @nhhagen and @hawi74, we discussed some problems with the current state of the library.

The library requires verbose initialization (many lines of code to start the datadog-collector)
The library is prone to errors: Middlewares needs to be initialized after the datadog-collector is started
The Tracer&Profiler is started separately from the Metrics-collector
The config is required to be read and parsed by the consuming library (from env-vars or env-files) and then passed to the library
Some functionality might still require direct calls to the underlying library (such as starting/ending manual spans)
Some parts have added null-safe checks (e.g. creating gRPC middleware when datadog is disabled), but others haven't (such as the metrics-collector), causing runtime-panics if the config is invalid.
When an application shuts down, datadog should attempt to flush any remaining traces/profiling/metrics before shutting down.

We discussed creating a v3 of the go-datadog-lib (or a new lib like go-monitoring-lib or go-telemetry-lib), or if we should only add a new way to bootstrap everything for datadog and deprecate the old methods.
We discussed 3 main ways of using the library:

As a config-only-lib (expect users to use dd-trace-go to manually start new traces/spans, and use contrib-middleware where needed)
As a fully encapsulated monitoring library (hide all implementations, wrap everything) (go-monitoring-lib)
Move to open-telemetry SDK instead of dd-trace-go, combined with one of the two above approaches (go-telemetry-lib)

The consensus from the meeting was that we should continue using dd-trace-go, but encapsulate more of the "read config and start collectors"-part, and ensure a better developer-experience for consumers.

We want to provide a single entrypoint to the library that reads the config from environment-variables.
We want to provide value-adding wrappers for existing middlewares where possible (good default-values, no-op middlewares if datadog is disabled, ensuring the datadog collectors is started)
We want to provide a better functionality when shutting down the application, to try to flush remaining traces/profiling/metrics.

We also want some small features/bug-fixes:

We want to automatically add the tracelogger hook to the global go-logger instance (for trace->log correlation)
We want to hijack the following log-message Datadog Tracer v1.68.0 WARN: Unsupported protocol "" in Agent URL "/var/run/datadog/apm.socket". Must be one of: http, https, unix. and swallow that message, as it is causing a lot of confusion for developers.

I think we should evaluate the option 3. Many problems that are highlighted would go away:

We want to provide a single entrypoint to the library that reads the config from environment-variables.

The entire open telemetry ecosystem is configurable using env: https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/. Effectively making it a zero-setup API, if one choses to do so.

no-op middlewares if datadog is disabled

Very important indeed, this is covered: https://opentelemetry.io/docs/specs/otel/metrics/noop/

ensuring the datadog collectors is started

No need if using OTLP. The OTLP protocol also allows to buffer/aggregate/filter the metric/traces/logs using a very flexible architecture:
https://opentelemetry.io/docs/collector/

trace->log correlation

Out of the box, especially now that logging is part of OTLP.

Good thoughts. I have treading a bit careful around open-telemetry in go, as it has been mentioned multiple times that they are known to make breaking changes.
See this Reddit-post from 2 years ago: https://www.reddit.com/r/golang/comments/yq3h5w/is_it_worth_instrumenting_with_opentelemetry/
And this from 1 year ago: https://www.reddit.com/r/golang/comments/18dn0ng/behavioral_breaking_change_proposal_in_otel_go/
It might be that they have matured more now.

@hadrienk and @tuhtah we also do not know how well the open-telemetry SDK works with Datadog. Replacing Datadog (the service) is not on the table as that is selected as the operational tool from the org.
For logging we still have to adhere to https://inventory.internal.coop/docs/default/component/guidelines/data_protection/#personal-data-in-logs

Personal Data in logs¶
Personal Data MUST NOT be written to system logs. System logs are not considered to be a part of the system since the logs are sent to a central separate system to enable the required observability for operational use. For more information see, Obeservability / Logging / Personal Data.

Good thoughts. I have treading a bit careful around open-telemetry in go, as it has been mentioned multiple times that they are known to make breaking changes.
See this Reddit-post from 2 years ago: https://www.reddit.com/r/golang/comments/yq3h5w/is_it_worth_instrumenting_with_opentelemetry/
And this from 1 year ago: https://www.reddit.com/r/golang/comments/18dn0ng/behavioral_breaking_change_proposal_in_otel_go/
It might be that they have matured more now.

Yes, I think that they took a 180 turn with some of the internals of the sdk at some point. But I wonder if this was also due to changes in the spec itself. The changes I've experienced previously were trivial I must say.
https://github.com/open-telemetry/opentelemetry-go/blob/main/VERSIONING.md

@hadrienk and @tuhtah we also do not know how well the open-telemetry SDK works with Datadog.

True, we would have to try it to be sure, but they are advertising it. The earliest mention is from one year ago.
One area that might be tricky is the mappings, but it seems well documented.

Replacing Datadog (the service) is not on the table as that is selected as the operational tool from the org.

Of course, using open telemetry does not mean replacing datadog.

For logging we still have to adhere to https://inventory.internal.coop/docs/default/component/guidelines/data_protection/#personal-data-in-logs

Personal Data in logs¶
Personal Data MUST NOT be written to system logs. System logs are not considered to be a part of the system since the logs are sent to a central separate system to enable the required observability for operational use. For more information see, Obeservability / Logging / Personal Data.

Not sure what you mean here, the logging part I'm mentioning is merely a logger API details (log, slog, logrus, zerolog etc). It would send the logs to the same place they are sent today (or somewhere else if we decide so).

Not sure what you mean here, the logging part I'm mentioning is merely a logger API details (log, slog, logrus, zerolog etc). It would send the logs to the same place they are sent today (or somewhere else if we decide so).

I haven't had a look at how OTEL does logging yet, but I assume it still supports logging via reading stdout/stderr, similar to how we do it today?

Not sure what you mean here, the logging part I'm mentioning is merely a logger API details (log, slog, logrus, zerolog etc). It would send the logs to the same place they are sent today (or somewhere else if we decide so).

I haven't had a look at how OTEL does logging yet, but I assume it still supports logging via reading stdout/stderr, similar to how we do it today?

It's pretty flexible, you can either export the logs using otlp, in that case there's no output. Or use a bridge to other structured log tech. I've tested with the new golang slog in a couple of other projects.

OpenTelemetry discussion moved to:

https://github.com/coopnorge/engineering-issues/issues/431


Fixed a bunch of warnings (from Eclipse)

Looks good, but one question: are there any alternatives to these serialVersionUID strings?
The aestheticist in me whimpers in pain...

not really, only alternative is not to be serializable ....

Gah, serializing and version numbering the exceptions seems kind of overkill...
Oh well, warnings are nasty, let's just merge this.

Done, thank you @vorburger !


Missing comment field in results

https://www.expectedparrot.com/content/f3ba4452-882f-4632-bff9-608356e57226

It's definitely there in the generated token string (see 'content'), which means it's not getting parsed correctly:

This seems to only be an issue on remote inference. Working on it.

Here's a succinct example
from edsl import QuestionMultipleChoice

q = QuestionMultipleChoice(
    question_name = "smallest_prime",
    question_text = "Which is the smallest prime number?",
    question_options = [0, 1, 2, 3]
)

results_remote = q.run()
results_local = q.run(disable_remote_inference = True)

It's a serialization issue:

@zer0dss did you have some bandwidth to investigate?

@johnjosephhorton Yes, I will check it.

@zer0dss Can you let me know when the edsl on coopr has this fix?

@johnjosephhorton The testing endpoint is update with the new version and the issue is fixed.


Add Heroku GIS buildpack

 Add PostGIS support to land-availability-lr app on Heroku using the proper buildpack

Codecov Report

Merging #11 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #11   +/-   ##
=======================================
  Coverage   30.96%   30.96%
=======================================
  Files          12       12
  Lines         155      155
=======================================
  Hits           48       48
  Misses        107      107

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 04eeba6...4247f62. Read the comment docs.

I'm new to heroku, but the docs for that buildpack mention you should also specify the underlying language's buildpack too e.g. heroku/python - does that apply to this case?

Python is automatically detected by the fact requirements.txt is in the root of the project

Great