WIP: model: scikit: Add support and usecase examples for MultiOutput models

Remove last commit(d7ef74d07368450d33f0923afb77fef87d4d118c)? i.e native accuracy scorer multi-output support.

Currently, we are trying to get mean of accuracies for the multiple labels which yields different output than the scikit scorer with same name.
I have 3 approaches, the other two commented in the file, and all of them give different scores. Probably the math is off with the other 2.
[ ] Will fix lint lines in case we keep the last commit.

test (., 3.7), test (., 3.8) and examples/dataflows.rst take forever.

[x] fixed by removing the numpy import.

[ ] Add exceptions of scorers for multioutput support in docs.

Codecov Report

Merging #1175 (8e725ba) into master (bbf4910) will decrease coverage by 0.16%.
The diff coverage is 37.50%.

@@            Coverage Diff             @@
##           master    #1175      +/-   ##
==========================================
- Coverage   86.32%   86.16%   -0.17%
==========================================
  Files         168      168
  Lines       10543    10558      +15
  Branches     1732     1739       +7
==========================================
- Hits         9101     9097       -4
- Misses       1100     1120      +20
+ Partials      342      341       -1

Impacted Files
Coverage Δ

dffml/accuracy/clf.py
40.54% <5.88%> (-21.00%)
:arrow_down:

dffml/accuracy/mse.py
86.20% <60.00%> (-5.80%)
:arrow_down:

dffml/util/python.py
93.33% <100.00%> (+2.42%)
:arrow_up:

dffml/util/data.py
72.92% <0.00%> (-2.77%)
:arrow_down:

tests/util/test_net.py
98.36% <0.00%> (-1.64%)
:arrow_down:

dffml/noasync.py
78.94% <0.00%> (-1.06%)
:arrow_down:

dffml/util/testing/consoletest/commands.py
75.56% <0.00%> (-0.06%)
:arrow_down:

dffml/source/df.py
76.54% <0.00%> (ø)

dffml/base.py
85.00% <0.00%> (+0.33%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bbf4910...8e725ba. Read the comment docs.

We decided that the accuracy scorers should take the features to score as their
last argument(s) to the score() method.
In this way, if a scorer wanted to work on multiple features, the method
signiture could be easily modified to be
    async def score(
        self,
        mctx: ModelContext,
        sources: SourcesContext,
        *args: Feature,
    ):

diff --git a/dffml/accuracy/mse.py b/dffml/accuracy/mse.py
index aeb00fb82..ed28d3d24 100644
--- a/dffml/accuracy/mse.py
+++ b/dffml/accuracy/mse.py
@@ -21,18 +21,17 @@ class MeanSquaredErrorAccuracyContext(AccuracyContext):
     Mean Squared Error
     """

-    async def score(self, mctx: ModelContext, sources: SourcesContext):
-        if len([mctx.parent.config.predict]) != 1:
-            raise InvalidNumberOfFeaturesError(
-                f"{self.__class__.__qualname__} can only assess accuracy of one feature. features: {features}"
-            )
+    async def score(
+        self,
+        mctx: ModelContext,
+        sources: SourcesContext,
+        feature: Feature,
+    ):
         y = []
         y_predict = []
         async for record in mctx.predict(sources):
-            y.append(record.feature(mctx.parent.config.predict.name))
-            y_predict.append(
-                record.prediction(mctx.parent.config.predict.name).value
-            )
+            y.append(record.feature(feature.name))
+            y_predict.append(record.prediction(feature.name).value)
         accuracy = sum(
             list(map(lambda x, y: abs(x - y) ** 2, y, y_predict))
         ) / len(y)

These changes are required to get scorers working with HTTP service.
We need to add a TODO to the HTTP service refactor to accpect the features
to score from the request body.
diff --git a/service/http/dffml_service_http/routes.py b/service/http/dffml_service_http/routes.py
index e11d82b3f..5aaa3a596 100644
--- a/service/http/dffml_service_http/routes.py
+++ b/service/http/dffml_service_http/routes.py
@@ -912,7 +912,11 @@ class Routes(BaseMultiCommContext):
         sctx_label_list = await request.json()
         sources = await self.get_source_contexts(request, sctx_label_list)
         return web.json_response(
-            {"accuracy": await actx.score(mctx, mctx.config.predict, sources)}
+            {
+                "accuracy": await actx.score(
+                    mctx, mctx.parent.config.predict, sources
+                )
+            }
         )

     async def api_js(self, request):
diff --git a/service/http/dffml_service_http/util/testing.py b/service/http/dffml_service_http/util/testing.py
index 4fa9e1cf8..4cc36cd0f 100644
--- a/service/http/dffml_service_http/util/testing.py
+++ b/service/http/dffml_service_http/util/testing.py
@@ -1,4 +1,5 @@
 import asyncio
+import tempfile
 import contextlib
 from http import HTTPStatus
 from unittest.mock import patch
@@ -63,7 +64,9 @@ class FakeScorerConfig:

 class FakeScorerContext(AccuracyContext):
-    async def score(self, mctx: ModelContext, sources: SourcesContext):
+    async def score(
+        self, mctx: ModelContext, feature: Feature, sources: SourcesContext
+    ):
         accuracy: int = 0
         async for record in sources.records():
             accuracy += int(record.key)
@@ -178,11 +181,20 @@ class TestRoutesRunning:

     @contextlib.asynccontextmanager
     async def _add_fake_model(self):
-        async with FakeModel(BaseConfig()) as model:
-            self.model = self.cli.app["models"][self.mlabel] = model
-            async with model() as mctx:
-                self.mctx = self.cli.app["model_contexts"][self.mlabel] = mctx
-                yield
+        with tempfile.TemporaryDirectory() as tempdir:
+            async with FakeModel(
+                FakeModelConfig(
+                    location=tempdir,
+                    features=Features(Feature("by_ten")),
+                    predict=Feature("by_ten"),
+                )
+            ) as model:
+                self.model = self.cli.app["models"][self.mlabel] = model
+                async with model() as mctx:
+                    self.mctx = self.cli.app["model_contexts"][
+                        self.mlabel
+                    ] = mctx
+                    yield

     @contextlib.asynccontextmanager
     async def _add_fake_scorer(self):

diff --git a/dffml/accuracy/mse.py b/dffml/accuracy/mse.py
index aeb00fb82..ed28d3d24 100644
--- a/dffml/accuracy/mse.py
+++ b/dffml/accuracy/mse.py
@@ -21,18 +21,17 @@ class MeanSquaredErrorAccuracyContext(AccuracyContext):
     Mean Squared Error
     """

-    async def score(self, mctx: ModelContext, sources: SourcesContext):
-        if len([mctx.parent.config.predict]) != 1:
-            raise InvalidNumberOfFeaturesError(
-                f"{self.__class__.__qualname__} can only assess accuracy of one feature. features: {features}"
-            )

I'm assuming here that these changes in mse were just to show how a scorer for multioutput would differ and not something to be implemented into mse as mse doesn't support multioutput and removing this logic and sending Features to mse results in the following error:

So I'm going to leave the logic as is, please let me know if we're not on the same page here @pdxjohnny .

Nice!!!
Great work @mHash1m! 💯 🥇 🌟


Include name of rules in RuboCop output

When I was working on clarifying style details in another PR, I had to enable two settings to make sense of it.
These are good settings, we should have them on always.

thanks! 🎉


Style: Remove preceding spaces from C# preprocessor-type keywords

Fix SA1006:PreprocessorKeywordsMustNotBePrecededBySpace StyleCop issues
PR Summary
Style fix:

Remove preceding spaces from C# preprocessor-type keywords

PR Checklist

[x] PR has a meaningful title

Use the present tense and imperative mood when describing your changes

[x] Summarized changes
[x] Change is not breaking
[x] Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
[x] This PR is ready to merge and is not Work in Progress.

If the PR is work in progress, please add the prefix WIP: to the beginning of the title and remove the prefix when the PR is ready.

User-facing changes

[x] Not Applicable
OR
[ ] User-facing Documentation needed

[ ] Issue filed - Issue link:

Testing - New and feature

[x] Not Applicable or can only be tested interactively
OR
[ ] Make sure you've added a new test if existing tests do not effectively test the code changed

[ ] Add [feature] if the change is significant or affects feature tests

CI-Windows  failed on Wait-FileToBePresent -File $errorFile -TimeoutInSeconds 10 in 'Access-denied test for removing a folder'.
Restarted.

@xtqqczze Thanks for your contribution!


added Arabic locale for Portal

I think this is perfect Arabic, however If someone can improve then I will be happy.

Thanks for the submission. I'm closing this for cleanup as we have these strings at this point.


feat(IconButton): add size and variant props

What does it do?
Add a size prop
So far the IconButton had only one size. For Review Workflows we need them to be large too, according to the design, to match the height of the Select component.
This PR adds support for the 3 sizes defined for regular buttons: S | M | L

Preview
Add a variant prop
So far IconButton had only one variant, which is very similar to the tertiary variant of Button: https://design-system-git-main-strapijs.vercel.app/?path=/story/design-system-components-button--variants
This PR adds the secondary variant, to be able to style the IconButton in the same way as Button (see screenshot).
In v2 we probably want to unify the variants across Button and IconButton, because the current defaults are different.
Preview

Note
This also replaces the inline snapshot for the modal, because with prettier@3 they can no longer be updated.

Summary of all failing tests
@strapi/design-system:  FAIL  packages/strapi-design-system/src/ModalLayout/__tests__/ModalLayout.spec.tsx
@strapi/design-system:   ● Test suite failed to run
@strapi/design-system:     Jest: Inline Snapshots are not supported when using Prettier 3.0.0 or above.
@strapi/design-system:     See https://jestjs.io/docs/configuration/#prettierpath-string for alternatives.
@strapi/design-system:       at saveInlineSnapshots (node_modules/jest-snapshot/build/InlineSnapshots.js:101:15)

Why is it needed?
See above.
How to test it?

https://design-system-git-feat-iconbutton-variants-strapijs.vercel.app/?path=/story/design-system-components-iconbutton--sizes

https://design-system-git-feat-iconbutton-variants-strapijs.vercel.app/?path=/story/design-system-components-iconbutton--variants

Refs: https://github.com/strapi/strapi/pull/17611

Can you rebase and add a changeset please? 🙏🏼

Yes I was waiting for that 👍🏼


could you please explain the use cases when should we use this component instead of using regular css media queries. thanks

@edorivai what you mentioned makes sense. I have a follow up question on how do we optimise for bundlesize? For example when my app is running on a small screen, ideally I shall only load SmallScreen and not load LargeScreen at all unless the screen size changes.

@kamleshchandnani I think regular bundle splitting with React.lazy() should work.


fix: next 10.0.8 changes loadConfig to be async

Fixes https://github.com/netlify/netlify-plugin-nextjs/issues/104
this means that sites using =>10.0.8 will be incompatible with plugin <= 2.0.1
BUT
sites using <=10.0.7 using plugin > 2.0.1 will still work at least 🙃

I think you implied this @lindsaylevine, but just to confirm:

Both next <=10.0.7 and next >=10.0.8 should work after this PR, since awaiting a non-Promise is valid.
next >=10.0.8 won't work without this PR, i.e. @netlify/plugin-nextjs <=2.0.1 won't work for them.


problem on vevo videos

vevo videos no longer opening

I'll second that..

also do not work SME Videos

i've tried many different solution but i haven't succeed

Yep, broken again for all kinds of videos like this.

i've looked some apps continue to work. there must be an another technique

@0xced Are you looking at this right now? I'll need to start looking into it in the next couple hours, but if you're already working on it, there's no sense in both of us doing it.
If you're busy, though, it'd be good for me to not expect an upstream fix soon.

I already started to look into it. The problem is that the execution of the player JavaScript fails with TypeError: undefined is not an object (evaluating 'window.navigator.j') before it can evaluate the parts required to descramble the signature. I’m not sure yet what is the most sustainable solution.

Cool. YouTube DL seems to be working without change, so something fundamental isn't different. Which is nice.

Signature function (Javascript)
function it(a) {
    a = a.split("");
    ht.De(a, 14);
    ht.po(a, 54);
    ht.PF(a, 1);
    ht.De(a, 37);
    ht.De(a, 61);
    ht.po(a, 68);
    return a.join("")
}

unscrambleSignature function (Objective-C)
JSValueRef unscrambledSignatureValue = JSObjectCallAsFunction(self.context, self.signatureFunction, NULL, 1, &scrambledSignatureValue, &exception);

exception
TypeError: undefined is not an object (evaluating 'ht.De')

Is it the reference error to the internal function?

I read through the JS, and ht.De(a, n) works out to "swap the nth and 0th characters of a", ht.po(a, n) works out to "reverse a, ignore n", and ht.PF(a, n) works out to "drop n characters off the front of a"
So it just scrambles it a little.
If I just translate that function as the body of the unscrambleSignature:, it mostly works.
One test legitimately fails, and another fails because it tests a broken signature function.
The issue is, I don't know how often that little script changes.
YoutubeDL also parses out and reads this function, but it does it using some terrible regular expressions and a switch statement as a really sketchy parser / tokenizer / interpreter.
On the other hand, it's working right now...
That's why they aren't having these issues.

I feel like it'd be nice if the JS issues were resolved, and this thing that had been working kept working.
If there's no fix by tomorrow, I'll try. Something.

As you noticed, youtube-dl has its own JavaScript interpreter, or rather a narrow subset of JavaScript. With JavaScriptCore available on iOS and OS X, there is no way I’m going down that route.
I have found a solution and I’m currently polishing it. I hope to make it available soon.

I just pushed a fix on the develop branch, please test if everything is OK. I will try to publish version 2.1.3 tonight (CEST).

very thnx @0xced @psycotica0

working now thx @0xced  @psycotica0

Works for me :) Thanks for the great work you do.

It was working.
Very thanks.

Looks good from here. Thanks!


Windows Phone 8.1 IE screen rotation intermittent glitch

Hi,
I've found an interesting problem with Bootleaf rendering on WP8.1 IE. Sometimes when I rotate the screen from portrait to landscape part of the screen doesn't get rendered. It happens approximately once out of three times. Refresh helps.

I've tested it on other browsers and other devices - works like charm. Freaking mobile IE!
Tried to workaround it myself - forcing to redraw the #container element - no luck.
I guess it's more a bug report for the Microsoft folks.

Yeah, not really sure what I can do about this... What version of IE? Does this happen on all Bootstrap sites? Have you tried rotating from this example page: this: http://getbootstrap.com/examples/starter-template/?

Yep, it's happens with the plain bootstrap as well. IE11 WP8.1.

I've reported this problem to the MS support.
https://connect.microsoft.com/IE/feedback/CreateFeedbackForm.aspx?FeedbackFormConfigurationID=5534&FeedbackType=1

¯\_(ツ)_/¯

Closing for now.


Duplicate IDs on ONS logo SVG icons used in header masthead and footer (WCAG Level A fail)

Need to set a param for the  id used in the <title> element for ons icons set in onsIcon(), so we can specify unique IDs in the header and footer.

Issue ID: DAC_Non_Unique_Ids_02
URL: https://ons-design-system.netlify.app/patterns/download-resources/examples/download-resources/
Page title: Service name
Journey: Journey 2, Step 2
The office for national statistics logo images on the page contains an id attribute. The value of the id attribute has been duplicated. This could cause complications for assistive technology reading the code. It may also cause a mismatch of content if the id is used multiple times.
Solution:
Id attribute values must be unique. Ensure that the office for national statistics logo images is revised to have unique id attribute values.

@yatesn  from page 18 of the June 2022 DAC DS report.


Migration<PHONE_NUMBER>5808 fails on existing database

The migration<PHONE_NUMBER>5808 fails on a system with existing data, with the following output:
15:43:26  DEBUG [69a86072] 	==<PHONE_NUMBER>5808 AddFiltersToEntitlements: migrating =========================
15:43:26 -- add_column(:entitlements, :filters, :text)
15:43:26  DEBUG [69a86072] 	rake aborted!
15:43:26 StandardError: An error has occurred, this and all later migrations canceled:
15:43:26
15:43:26 PG::DuplicateColumn: ERROR:  column "filters" of relation "entitlements" already exists
15:43:26 : ALTER TABLE "entitlements" ADD "filters" text/opt/manageiq/shared/bundle/ruby/2.3.0/bundler/gems/rails-e0a9baa04402/activerecord/lib/active_record/connection_adapters/postgresql/database_statements.rb:98:in `async_exec'

Not sure if the given information is enough to reproduce/fix. if needed, I can produce a DB dump with the contents of the DB.

Commenting out the add_column on the db/migrate/20160428215808_add_filters_to_entitlements.rb migration causes the following:
$ cd /opt/manageiq/releases/20160502160503 && ( export RAILS_ENV="production" ; bundle exec rake db:migrate )
fatal: Not a git repository (or any parent up to mount point /opt/manageiq)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
DEPRECATION WARNING: `config.serve_static_files` is deprecated and will be removed in Rails 5.1.
Please use `config.public_file_server.enabled = false` instead.
 (called from block in <top (required)> at /opt/manageiq/releases/20160502160503/config/environments/production.rb:13)
==<PHONE_NUMBER>5808 AddFiltersToEntitlements: migrating =========================
==<PHONE_NUMBER>5808 AddFiltersToEntitlements: migrated (0.0009s) ================

==<PHONE_NUMBER>5825 MoveFiltersToEntitlements: migrating ========================
-- Moving MiqGroup filters to Entitlements
rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

undefined method `filters' for #<MoveFiltersToEntitlements::MiqGroup:0x000000086c65f8>/opt/manageiq/shared/bundle/ruby/2.3.0/bundler/gems/rails-e0a9baa04402/activemodel/lib/active_model/attribute_methods.rb:433:
in `method_missing'

So three migrations were actually moved, we may need to watch out for issues with each of them
https://github.com/ManageIQ/manageiq/pull/8337/commits/bb822f93284347c049e00971118412a6ff83a329


Butterfly Aspect

Changes / Comments

add some variable to mage rolls
add new settings to magic
add an initiative button

Roll20 Requests
Comments are very helpful for reviewing the code changes. Please answer the relevant questions below in your comment.

[ ] Does the pull request title have the sheet name(s)? Include each sheet name.
[ ] Is this a bug fix?
[ ] Does this add functional enhancements (new features or extending existing features) ?

Character Sheet Info Roll20 Internal Use only.


Can't run dotnet command after install

Problem encountered on https://dotnet.microsoft.com/en-us/learn/dotnet/hello-world-tutorial/install
Operating System: macos
Opened a new Terminal window after installing intel version, getting following error:

machine details:

Did you install the SDK? If so, which version? And did you restart the terminal after the installation?

yes, I attempted the installation and it said it was installed successfully, but running the dotnet command in a new terminal window still gave the error.

I did restart the Terminal application too, to be clear, not just a new window.

installed the .NET SDK x64 (Intel) version

Can you check if you see dotnet installed on the install folders as shown at https://learn.microsoft.com/en-us/dotnet/core/install/how-to-detect-installed-versions?pivots=os-macos#check-for-install-folders
Maybe dotnet is not on the path.
/cc @eiriktsarpalis

Looks like they exist, could the multiple versions be causing the issue?

Hi @conoremcd - can you try running the dotnet binary located at /usr/local/share/dotnet/dotnet directly first to verify if the actual binary is executable? That would help debug any architecture-specific issues going on.
If that works, the next guess I'd have is that the symlink in (I think) /usr/local/bin wasn't created. There should be a symlink in that folder (or one of the other common symlinked-binary folders on macOS) that points to /usr/local/share/dotnet/dotnet. If there isn't then the installer may have had an issue. I believe the installer logs to the system log, which I think is /var/logs/install.log - is there anything interesting/relevant there?

Hi @baronfel getting a command not found when attempting to run the dotnet binary so it seems thats not actually executable for some reason. I also took a look into the install.log file and didn't see anything that stood out to me as an issue

@conoremcd is dotnet an exectuable? if you use the file command to inspect it, what's the output? That kind of command not found response is what I'd expect to see if you had an executable for a different architecture - e.g. accidentally installing an arm64 version of the SDK.

Looks like it is an executable?

I'm 100% sure I installed the intel version, but should I attempt an uninstall and just try again to be sure?

It doesn't hurt @conoremcd

Have you tried uninstalling and installing again?

I did and it still did not work

What's on your PATH (run export)?
What might help is to add soft link to dotnet:
lrwxr-xr-x  1 root  admin  30 Mar 31 12:25 /usr/local/bin/dotnet -> /usr/local/share/dotnet/dotnet

That's how I have it on my mac and I think I added it myself manually.


NoSuchMethodError

I have ElasticSearch 1.5.0 and fsriver 1.3.1...after I install the plugin using plugin -install, when I attempt to run elasticsearch.bat i get an Initialization failed error:
NoSuchMethodError[org.elasticsearch.rest.BaseRestHandler.(Lorg/elasticsearch/common/settings/Settings;Lorg/elasticsearch/client/Client;
Should I be using a different version of fsriver for this version of elastic search?

Yes. This version is not compatible with 1.5.0 (and with 1.4.x BTW)
Will be fix with #108. No ETA though.

So the only solution right now is to go back on ES version? Or will building from source make it work with 1.5?

Rivers has just been announced as deprecated BTW: https://www.elastic.co/blog/deprecating_rivers

But yes you can fork the code and Patch it. Look at #100


a new created node after updating is not able to reach the t1 edge.

Describe the bug
After redeployment/updates of controlplanes or workers, new created vm's get directly the same ip-address of the deleted old vm (because that is the first free ip in the pool). This can create some issues because most of the time the given templates from vmware don't send any arp requests that they are using the already used ip. NSX-T by default discovers new ip's after a time-out from 8 minutes (the arp table in nsx-t doesn't get directly updated). because of the arp issues the vm cannot ping the t1 router, because of this it can also not reach the  api and join the cluster.
So far i have been able to activate the retry join option to the cluster, but this makes the vm creation time arround 10 minutes.
The second solution was to add a arping after booting the vm. now vm creation times are arround 2-3 minutes what is acceptable for me.
i noticed that after running the command: “arping -U -I ens192 <VM_SELF_IP_address> -c 3" the tier 1 router becomes responsive. As a fix i have added this command in the template. this brings the creation time to a steady 2-3 minutes all the time.
Reproduction steps

updating nodes to a new image
after one node is deleted and a new node is being created with the same ip as the deleted node.
the new node cannot reach the tier 1 gateway. because of this it cannot join the cluster.
...

Expected behavior
The new created node should be able to reach the gateway directly and lower down the creation time.
Additional context
The error that can be seen when loadbalancer is not reachable: “[preflight] Running pre-flight checks
[2023-05-26 14:59:39] error execution phase preflight: couldn’t validate the identity of the API Server: Get “https://****:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s”: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
[2023-05-26 14:59:39] To see the stack trace of this error execute with --v=5 or higher”

The specific fix as root-caused by @TimTinneveld is to have the following in the cloud-init file:
arping -U -I ens192 <VM_SELF_IP_address> -c 3


Second user in contributors can't submit.

Hi,
I'm trying to submit HaxePunk 2.5.4 but it isn't working for me,
in haxelib.json we have "contributors": ["heardtheword", "_ibilon"], and it was pushed to haxelib for the previous version (2.5.3, last in semver order).
But I get User '_ibilon' is not a developer of project 'HaxePunk'.
Is there anything else that should have been done or is it a bug?

That is weird. Just tried it myself and it works.
But apparently in your the database was not updated correctly, because according to the server you are not a contributor of HaxePunk: http://lib.haxe.org/u/_ibilon
Would you be so kind as to have the update be submitted by heardtheword to see if that updates you project list correctly?

Submitted. I'm pretty sure we've done this in the past. Can you double check that _ibilon is listed as a contributor now?

Nope. I've checked the code and this is quite a silly bug. Haxelib only considers the project changed if the number of developers changes: https://github.com/HaxeFoundation/haxelib/blob/master/src/tools/haxelib/Repo.hx#L191
Since you are replacing "ibilon" with "_ibilon" the code assumes there are no changes. For now, can you please just resubmit (the same version) with ["heardtheword", "ibilon", "_ibilon"] (and then follow up with ["heardtheword", "_ibilon"] if you want to)?

Done. Thanks for looking into this.


ImportError: cannot import name 'ExecutionResult' from 'graphql'

Im trying to import "from starlette_graphene3 import GraphQLApp" but this error throws me.
Traceback (most recent call last):
File "/usr/local/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/local/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.8/site-packages/uvicorn/subprocess.py", line 61, in subprocess_started
target(sockets=sockets)
File "/usr/local/lib/python3.8/site-packages/uvicorn/server.py", line 49, in run
loop.run_until_complete(self.serve(sockets=sockets))
File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/usr/local/lib/python3.8/site-packages/uvicorn/server.py", line 56, in serve
config.load()
File "/usr/local/lib/python3.8/site-packages/uvicorn/config.py", line 308, in load
self.loaded_app = import_from_string(self.app)
File "/usr/local/lib/python3.8/site-packages/uvicorn/importer.py", line 23, in import_from_string
raise exc from None
File "/usr/local/lib/python3.8/site-packages/uvicorn/importer.py", line 20, in import_from_string
module = importlib.import_module(module_str)
File "/usr/local/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 848, in exec_module
File "", line 219, in _call_with_frames_removed
File "/app/./src/main.py", line 3, in
from starlette_graphene3 import GraphQLApp
File "/usr/local/lib/python3.8/site-packages/starlette_graphene3.py", line 17, in
from graphql import (
ImportError: cannot import name 'ExecutionResult' from 'graphql' (/usr/local/lib/python3.8/site-packages/graphql/init.py)

Hi @jrdnbrj,
Do you have Graphene version 3.x (not 2.x) installed?

@ciscorn
None of those, I have version 0.3.2

@jrdnbrj Try the following command in your terminal:
pip3 install starlette-graphene3 graphene==3.0b7 -U


Fix bug where - was passing as a valid date.

Fix https://github.com/pydantic/pydantic/issues/7872
- is no longer parsing as a valid date.

replaced by #52.


Add Java 17 to test matrix

Needs someone with write permission to pretest.

Superseded by #321. Thanks though!


RawValue.ONE & RawValue.ZERO being instance objects could cause severe problems!!!

RawValue.ONE.negate() no longer modifies self. Instead it now returns a new object of type RawValue.


[Bug]: Invalid environment variables

What happened?
Hey team,
I am just starting to install your project and followed the README.md but before the last step when I have to go to 'http://<IP_ADDRESS>:3000/api/webhooks/run-jobs?token=WEBHOOK_TOKEN&run=update-metrics' I get this error:
Invalid environment variables:
 CLICKHOUSE_TRACKER_URL: Invalid url
 SCHEDULER_ENDPOINT: Invalid url
 ORCHESTRATOR_ENDPOINT: Invalid url

I've check your issues and it is not the first time something like this happens, apparently you fix it by letting them being optional but it does not seems to be the case anymore
I am on the latest commit
Steps to reproduce the problem

git clone https://github.com/civitai/civitai.git # I'm on commit 8c940ebd5b27b0ecaeb889b0243dd5e625ce9a52
cd civitai
npm install # version 10.1.0
docker compose up -d
cp .env-example .env
nano .env # edit S3_UPLOAD_KEY and S3_UPLOAD_SECRET with the freshly created key/secret on http://<IP_ADDRESS>:9001/access-keys/new-account and set WEBHOOK_TOKEN with random characters
npm run db:migrate
npm run db:generate
npm run dev

Then go to http://<IP_ADDRESS>:3000/api/webhooks/run-jobs?token=WEBHOOK_TOKEN&run=update-metrics and error occurs
What should have happened?
Start the metrics update job I guess ^^
What platforms do you use to access the site?
Linux
What browsers do you use to access the site?
Google Chrome
Additional information, context and logs
Here is the full log of the command npm run dev:
><EMAIL_ADDRESS>dev
> next dev

ready - started server on <IP_ADDRESS>:3000, url: http://localhost:3000
info  - Loaded env from /home/user/civitai/.env
warn  - You have enabled experimental feature (largePageDataBytes) in next.config.mjs.
warn  - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use at your own risk.

Browserslist: caniuse-lite is outdated. Please run:
  npx browserslist@latest --update-db
  Why you should do it regularly: https://github.com/browserslist/browserslist#browsers-data-updating
event - compiled client and server successfully in 14.5s (8400 modules)
Attention: Next.js now collects completely anonymous telemetry regarding usage.
This information is used to shape Next.js' roadmap and prioritize features.
You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL:
https://nextjs.org/telemetry

wait  - compiling /api/webhooks/run-jobs (client and server)...
event - compiled successfully in 3.7s (1178 modules)
❌ Invalid environment variables:
 CLICKHOUSE_TRACKER_URL: Invalid url
 SCHEDULER_ENDPOINT: Invalid url
 ORCHESTRATOR_ENDPOINT: Invalid url

wait  - compiling /_error (client and server)...
error - (api)/src/env/server.mjs (13:8) @ eval
error - Error: Invalid environment variables
    at eval (webpack-internal:///(api)/./src/env/server.mjs:19:11)
  11 | if (!_serverEnv.success) {
  12 |   console.error('❌ Invalid environment variables:\n', ...formatErrors(_serverEnv.error.format()));
> 13 |   throw new Error('Invalid environment variables');
     |        ^
  14 | }
  15 |
  16 | for (const key of Object.keys(_serverEnv.data)) {
event - compiled client and server successfully in 807 ms (9004 modules)

i guess the main branche it always under dev.
you can try to play with the code
in src\env\schema.mjs edit:
CLICKHOUSE_TRACKER_URL: z.union([z.string().url().nullish(), z.literal("")]),
SCHEDULER_ENDPOINT: z.union([z.string().url().nullish(), z.literal("")]),
ORCHESTRATOR_ENDPOINT: z.union([z.string().url().nullish(), z.literal("")]),

and in your .env set any value for ORCHESTRATOR_ENDPOINT:
ORCHESTRATOR_ENDPOINT=http://localhost:3000/
you can now access to the site at : http://localhost:3000/posts because the home page (http://localhost:3000) doesnt work i guess there is bug when loading the home page blocks when the table is empty
else you can try other branche

Confirmed this error on Ubuntu 22.04, using the setup instructions in git as of today

Solution here:
https://github.com/civitai/civitai/issues/544#issuecomment-1836962755

I confirmed this error is happening on latest master. @NicksonYap, your commit hash mentioned in the post still has the same problem but with a different url

can't we include the clickhouse docker stuff so it's out of the box?

I had the same issue after I rolled back to https://github.com/civitai/civitai/commit/f5687d449f1be6200a2b209c38a04a0b52636815.
Just for good measure, I did recreated a copy of .env_example, filled in the S3 upload key, secret and the web token, and commented out SCHEDULER_ENDPOINT and GENERATION_ENDPOINT.
I was able to at least get it started after this.

@advaithhl  - thanks for pointing this out -
yes - it's working. thanks again.

Can't seem to get it even with the stripe info and the scheduler and orchestrator disabled. every time I fix an error there are 2 more that pop up. Usually, it has dependencies that differ or version mismatches or this ver installed but it's reading a different ver. But this is ALL APIs that need to be set up and figured out i don't even know what some of them do : /
If any of you got it going and can help me out, at least if we knew which  APIs we have to set and which we can ignore and which to omitt.

goto roll back to this commit -
https://github.com/civitai/civitai/commit/f5687d449f1be6200a2b209c38a04a0b52636815
then comment out the scheduler as above screenshot


Cover more sections of the template

At present the plugin only replaces pseudo parameters in the Resources section of the template.
#12 suggests adding support for replacing pseudo parameters in more sections.
This extends the existing recursive parsing of the template to perform replacement across the entire template, thus replacing parameters across all sections.
It is possible this goes too far in replacing the parameters, so it could be revised to only perform replacement in certain sections.
This also adds tests for the Outputs section of the template.

Awesome!


setTableProps ignored in prod

When enabling dense table in my local with setTableProps as per example all works as expected. However when released the styles are ignored. The only difference seems to be when I have the env as production (although when in watch mode and production it works) bundling with parcel js.
Anyone else seen this?
Here you can see it working as expected:
https://codesandbox.io/s/happy-bhaskara-ft2vo
And here is a a small repo to show it working on yarn start but fails on yarn build then yarn serve
https://github.com/wroughtec/tableDemo
It seems to be an issue with production and parcel as the first version I tried with CRA exactly the same code only difference was the scripts worked.

Tech
Version

Material-UI
4.7.0

MUI-datatables
2.13.0

React
16.12.0

Hey @wroughtec, not sure offhand what the issue might be, but I can offer this bit of advice for debugging, which has definitely bitten me before. When using semver to manage your packages, it's possible that you have different versions of dependencies running in production vs local. I would check your production dependency versions against your local versions and see if that's the case. Any upgrade on any dependency might have broken something. If I'm on a project where I need to be extra cautious about this happening, I will stop using semver in my package files, and make sure to use only exact versions.

I wish that was the case but the small repo I set up shows the issue locally comparing dev with prod. I also used CRA with the same dependencies and that had no issue so I suspect its something Parcel is doing.

@wroughtec Your example didn't have a setTableProps option on the options object. Also, while looking at the Material UI docs, I noticed that they recently removed the "dense table" property for the Table component. If your prod and dev versions are using different versions of the lib (like @gabrielliwerant suggested), that may be the source of the problem. Have you tried setting other properties on the Table object to see if those carry over?
The way setTableProps works is that it attaches the props you give it to the Table component, so it seems odd that Parcel would be interfering. I updated your example so that it sets the style property of the Table component:
https://codesandbox.io/s/upbeat-breeze-f9fr7
You may want to try something like that and see if your prod version gets correctly updated.

@patorjk It does on the repo example:
https://github.com/wroughtec/tableDemo/blob/c79606ca100dfb60f8c95716b7ec42b04dc53f6e/src/index.js#L141
Must have not saved the codesand does now
https://codesandbox.io/s/happy-bhaskara-ft2vo
I can confirm they are not using different versions of the lib as I had it running locally both dev and prod builds gave different results same libs and when I tried with a webpack build had no issues either.
i have updated the example to prove this:
https://github.com/wroughtec/tableDemo
All you need to do is install
yarn
then run
yarn serve
You can rebuild if you wish as well check the readme for info. I also locked down versions to further prove that I am not going mad ;)

https://codesandbox.io/s/github/wroughtec/tableDemo
It seems to work fine. Is there something that is missing from how CodeSandbox executes that would affect things?

@patorjk that is my whole point it works fine in dev (codesandbox run as dev). The repo shows the differences comparing webpack and parcel builds when in production.

I'm having the exact same issue with a datatable that has setTableProps: () => ({ size: "small" }):
Here's the same table with two different build options:
Step 1: rm -rf node_modules ; rm -rf .cache ; parcel build

Step 2: rm -rf .cache ; parcel watch

So I setup a create-react-app project and installed mui-datatables, and these options seem to work fine:
const options = {
      filter: true,
      filterType: 'dropdown',
      responsive: 'simple',
      setTableProps: () => (
        {
          style: {
            backgroundColor: 'red'
          },
          size:'small'
        }
      )
    };
In my production setup at work (which uses a fork of this table - though it has the setTableProps feature), it also seems to work. If you right click and inspect the table element, is the property not there?

No, the small property is nowhere to be found. But that's not actually a html property though.

If you input this:
  setTableProps: () => {
    console.log('setting table props');
    return {
      style: {
        backgroundColor: 'red'
      },
      size:'small'
    };
  }

Do you see the console statement in dev, but not production?

I see it in both cases. However in the dev build (parcel watch), there is a css class MuiTableCell-sizeSmall applied, but I cannot find the corresponding rule in the prod build (parcel build).

What if you add a "styling" property or some other property? Do you see it on the tag when you inspect in production? Also, what happens if you add a non-mui class like "test_class" to the table, does that appear?

		setTableProps: () => {
			return {
				size: "small",
				style: {
					backgroundColor: "red",
				},
				class: "foobar",
			};
		},

property
parcel build
parcel watch

size
✗ no rule with small padding
(✓) class exists

style
✓
✓

class
✓
✓

So I cloned the repo from @wroughtec and can confirm size doesn't show up, but it looks like the table is applying the dense styling. If I inspect the "td" components of the table, and then toggle the "Dense Table" option, I can see styling added and removed.
I'm thinking this is a material-ui thing. The setTableProps method is applying props to the Material UI Table component. I'm assuming this component reads in the "size" prop but doesn't apply it to the HTML table tag. It probably only passes on props that aren't apart of its API. One data point in support of this idea is if I add a component field:
	setTableProps: () => {
		return {
			size: "small",
			style: {
				backgroundColor: "red",
			},
			className: "foobar",
                            component: "table"
		};
	},

That doesn't get added to the table tag either. So I'm thinking everything is actually working correctly. Though I'm curious if you see any styling changes on the "td" tags. A "MuiTableCell-sizeSmall" class gets added to td tags when I set "size" to "small".

it looks like the table is applying the dense styling

It doesn't for me. In the production build, there are no MuiTableCell-xyz classes, only jss-123. And none of those have the dense padding values.

What occurs if you put a normal table below your datatable and manually put a size field on it:
import Table from "@material-ui/core/Table";
import TableRow from "@material-ui/core/TableRow";
import TableCell from "@material-ui/core/TableCell";

return (
    <>
        ...your datatable here...

            <TableRow>
                <TableCell>Test</TableCell>
            <TableRow>

    </>
);

Does the table have any different classes than the datatable? It look like the size field puts classes on the "td" element. Are there any class differences between the td components?

Test table:

Datatable:

The inline styles are my current workaround for the issue (setCellProps).

I'm running in the same problem: running with parcel watch works as expected, parcel build doesn't add the MuiTableCell-sizeSmall class and thus doesn't apply my custom theme styling for small table cells.

Setting the size on the TableCell (<TableCell size="small">...</TableCell>) itself works and adds the MuiTableCell-sizeSmall class to the  element
Setting the size on the Table (
...
) does NOT work on builds with parcel. It works on development builds as expected, same as @AlCalzone and @wroughtec described before.

So, seems that in production builds the props aren't passed down to the cell properly for some reason. Any ideas?

I'm running in the same problem: running with parcel watch works as expected, parcel build doesn't add the MuiTableCell-sizeSmall class and thus doesn't apply my custom theme styling for small table cells.

Setting the size on the TableCell (<TableCell size="small">...</TableCell>) itself works and adds the MuiTableCell-sizeSmall class to the  element
Setting the size on the Table (
...
) does NOT work on builds with parcel. It works on development builds as expected, same as @AlCalzone and @wroughtec described before.

So, seems that in production builds the props aren't passed down to the cell properly for some reason. Any ideas?


children of the emitter and a bug

working with more than one child emitters (cutting/deleting) crashes the editor when playing the particle (shift), if saved before crashing the particle file can't be opened again

pasting a child emitter to a parent, closing-reopening the file sometimes puts it to a false parent emitter

child emitter layer is behind the parent emitter layer when played in the particle editor, however is always in front of the parent emitter when played ingame. This might be an exporting issue. I'd greatly appreciate if the editor would export the emitter layer order just like it's seen in the editors preview (always parent in front of child) to achieve the same effect seen on this picture - from the editor - ingame

Appearance - Fixed random rotation: every time a particle file is reopened in the particle editor, the second (+-) value gets down by one. It's really annoying, when you're constantly changing (thus reopening) files you work with.

2/3 are Possible duplicate of #2

@Megabalta, for items 1 and 2, can you download the latest build from the project page and give exact reproduction steps to get a crash or incorrect behavior?

@MikeLankamp it seems to be working right now, although I couldn't test pasting child emitters due to 1.5.b3.
v1.5 (32 bit) has 5 bugs I've discovered btw.
1.5.b1. won't remember game directory, needs input at every start
1.5.b2. description says add EaW-Forces of Corruption directory, but to display textures the program really needs the EaW game directory/GameData directory, so not that of the expansion (in Empire at War Gold)
1.5.b3. missing right click menu on the emitters panel, which makes pasting child emitters impossible (no other menu has the option)
1.5.b4. missing icon textures (for new emitter, delete emitter etc.) above Basic-Appearance-Physics tabs
1.5.b5. missing icons for emitter types (bursts, continuous stream, weather) in emitter panel

I already fixed bugs 1.5b4 and 1.5b5 (missing icons).
This issue has now become a collection of several bugs. It's generally good practice to have one issue per bug, otherwise it gets hard to track what's happening, so I'm going to close this one for that reason.
@Megabalta, please create a new issue for each of the remaining new bugs, as well as for the original bugs in this issue that haven't been fixed yet (i.e. 1, 2 and 4; 3 indeed seems a duplicate of #2).


Layout of datashaded curves breaks when viewing html

Hello, and thank you for your time. I've been using datashading with great success at my work; however, I've recently begun to try incorporating holoviews to interact with the graphs. I am not using a notebook, which I believe is the root of the issue. I have a dataframe with four columns. 0th column is time, columns 1,2,3 are x,y,z respectively. Each column consists of multiple 'runs' of the data, separated by a row of NaN. I am creating curves from each column, putting them in a layout, and then trying to save to datashaded layout. Here is my code:
    hv.extension('bokeh')
    curvesx = hv.Curve(df[['0','1']])
    curvesy = hv.Curve(df[['0','2']])
    curvesz = hv.Curve(df[['0','3']])

    layout = curvesx * curvesy * curvesz
    renderer = hv.renderer('bokeh').instance(fig='html')
    renderer.save(datashade(layout), "html/"+data)

When I open the html file, nothing shows up. Once I select developer tools, then the datashaded graph appear along with the error:

Jupyter notebooks comms not available. push_notebook() will not function. If running JupyterLab ensure the latest jupyterlab_bokeh extension is installed. In an exported notebook this warning is expected.
_ @ bokeh-0.13.0.min.js:31
VM50:57 Uncaught TypeError: Cannot read property 'get_client_comm' of undefined
at t.eval (eval at get (bokeh-0.13.0.min.js:31), :57:34)
at t.execute (bokeh-0.13.0.min.js:31)
at t. (bokeh-0.13.0.min.js:31)
at e.t.emit (bokeh-0.13.0.min.js:31)
at e.emit (bokeh-0.13.0.min.js:31)
at t.u._setv (bokeh-0.13.0.min.js:31)
at t.u.setv (bokeh-0.13.0.min.js:31)
at t._layout (bokeh-0.13.0.min.js:31)
at e.t._layout (bokeh-0.13.0.min.js:31)
at e.t._do_layout (bokeh-0.13.0.min.js:31)

I didn't believe I was using anything related to a notebook in my code. I believe this to be related to the holoviews renderer otherwise I can post this to the bokeh git, and please forgive me if this is a mistake on my end. Without datashading the layout, or the curves independently, this creates an html file perfectly.
Thank you

renderer.save will normally export a visualization to a static HTML file, without any Python process attached to it for running Datashader.  Luckily, Bokeh server allows you to set up such a pairing and a communication channel to replace the missing Jupyter comm listed in the error above.  You can start with the examples in http://pyviz.org/tutorial/13_Deploying_Bokeh_Apps.html .  E.g. https://github.com/pyviz/pyviz/blob/master/apps/osm-1billion.py should work as a starting point; just replace the HoloViews object returned by make_view with your own layout object, then do "bokeh serve osm-1billion.py".

Wow, thank you for the quick response!
Is running an instance of the server the only way to graph a datashaded layout then? Is that needed for interacting with the graph and re-aggregating the data? Makes sense why using a notebook is so popular for this then. Booting a server and running it via bokeh doesn't really fit into our workflow honestly, but if needed I can do that

You can export a static rendering of any datashaded plot by just passing dynamic=False to datashade(); the resulting plot should display properly without any Python process.  But you do need Python if you want  the plot to update when you zoom in; what's embedded in the initial plot is just a rasterized image of the data, with no access to the data itself unless Python is available.  You can fake some limited support for zooming by specifying a larger height and width to the datashade() call, causing the higher resolution plot to be embedded in the HTML file, but that will quickly generate quite a large HTML file...

That addition works perfectly, thank you! I totally forgot about setting dynamic in like that. You can mark this issue as closed if needed. Thanks for the help, and hopefully this helps anyone else stuck like me!
Adam

Reopening, you shouldn't have to use dynamic=False. I think this needs a small patch in pyviz_comms.

@jlstevens do you need information from me? or was this directed at @philippjfr ?

That was for me, I'll make a PR this weekend.

pyviz_comms has been fixed for a while.


Add some more commands.

The failure is due to https://github.com/Manishearth/rust-clippy/issues/1476


Update Portfolio “2023-12-27-primeiro-post”

Automatically generated by Netlify CMS

👷 Deploy Preview for driveclean-teste processing.

Name
Link

🔨 Latest commit
9050e2d7a907b79d2196052f174122e94b8ecfdb

🔍 Latest deploy log
https://app.netlify.com/sites/driveclean-teste/deploys/658c0976d0e56500082cb362


Fix trace arg name

Fix trace arg name
Description
Resolves: #193
Checklist

[x] Linked to Github Issue

Codecov Report

Merging #202 (0c897c4) into main (cf81622) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #202   +/-   ##
=======================================
  Coverage   94.87%   94.87%
=======================================
  Files          26       26
  Lines        4816     4816
=======================================
  Hits         4569     4569
  Misses        247      247

Impacted Files
Coverage Δ

src/main.rs
0.00% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cf81622...0c897c4. Read the comment docs.


refactor: Remove liquid templates

I noticed when working on #352 that the Liquid decency is outdated and flagged in a security issue. This PR refactors the preview site templates to use a simpler approach as we're not using any of the features Liquid provides.

Thanks @abovedave for the PR. I am all up for simplifying but consider that we have recently introduced liquid templates for mocking Core's layout in new Free Refills services. I don't see our usage of liquid going away any time soon.
Perhaps there is another webpack loader we can use instead of removing the liquid templates all together?
Where can I read more about the vulnerabilities of that loader?
Thank you

Ah I see! I could just swap out the new Function() with the actual Liquid package and skip the unmaintained loader if we really want it? It just seemed uncessary to use liquid just for a simple layout-view setup.
This was the error:
# npm audit report

liquidjs  <10.0.0
Severity: moderate
liquidjs may leak properties of a prototype - https://github.com/advisories/GHSA-45rm-2893-5f49
No fix available
node_modules/liquidjs
  liquidjs-loader  *
  Depends on vulnerable versions of liquidjs
  node_modules/liquidjs-loader

It just seemed uncessary to use liquid just for a simple layout-view set

@abovedave I would like to talk to Dan before removing Liquid all together from this repo. If the new Function() approach can buy us some time and fix the vulnerability I would prefer it. Thanks 🙏

I haven't reviewed this PR yet (expect a review later today), but I'm 100% ok with dropping liquid in this repo. The reason I introduced it was because I wanted a simple shared layout for the docs and liquid syntax was already used in Stacks's docs. The use of liquid in this repo is probably overkill, but it worked ok and stayed out of the way. If it is causing issues now, then let's ditch it.

@b-kelly If we are going to keep a templating language of sort, I would prefer to stay consistent with what we use in Stacks docs. We have just started to use liquid also for some free refills mock layout stuff (see WIP). Perhaps it is just a matter of swapping the existing loader with one that does not have vulnerabilities. I haven't looked in details but maybe this one could be an option.

@giamir I'm ok with adding back in a special loader or templating language, but I think it's overkill for our use case. 100% of the code that does the "templating" in this case is just a single line:
htmlLayout.replace("<%=content%>", content)

So... we really don't need a full blown templating solution for this. Imo, we can revisit in the future if we ever need a more full-featured solution.


3D character in body

Hi,
I have used your sample code to get emails from an inbox. But, the body has 3D characters all over the markup, hence the mail not rendering properly.
e.g: <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DUTF-8">
I have already tried .toString('utf8'), but makes no difference.
Any ideas?

This seems related to #16. I have not dug into the issue to determine the cause.

FYI: I have just tested this with the node-imap package directly, and I get the same issue.


Update license and README

License should be updated to remove the non-affiliation disclaimer as Colore is now developed in partnership with Razer.
Similarly, the README should also be updated to reflect this.

This was fixed in #131


Add diagnostic for pip.conf

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.

[x] Have you successfully run brew typecheck with your changes locally?
[ ] Have you successfully run brew tests with your changes locally? All failing tests are unrelated, see also #9170.
[ ] Have you successfully run brew man locally and committed any changes?

Customizing pip may break the python formula and cause all sorts of mayhem.
Specifically, it may break the formula’s postinstall (which relies on pip install) and any other established workflow where pip install is involved.
The new diagnostic detects the presence of Pip configuration files at known locations and prints a warning.
Thanks @sjackman for suggesting this PR. Also thanks to @Rylan12, @dtrodrigues and @alebcay for helping figure out the problem.

Review period will end on 2021-01-08 at 19:37:37 UTC.

Review period will end on 2021-01-08 at 19:37:37 UTC.

Is there any way of overriding the paths that we look at here (or at least those in $HOME) so we can avoid pip looking at user configuration entirely?

@MikeMcQuaid I looked at pip’s implementation and documentation.
One of the paths (~/.config/pip) can be overridden by the user if they set XDG_CONFIG_HOME. I didn’t take that into account as Homebrew filters that environment variable out anyway (I think).
The other paths are all hard-coded, and I found no obvious method to keep pip from looking at those.
Another way to bend pip to our will is to add --no-user to all places where Homebrew invokes pip install. (That includes e. g. postinstall in python and our poet snippet.)
@Rylan12 and I have discussed and abandoned that alternative. We feel it’s probably not worth the effort.
After that discussion, @sjackman suggested – as a middle ground – what ended up becoming this PR.

Is there any way of overriding the paths that we look at here (or at least those in $HOME) so we can avoid pip looking at user configuration entirely?

@MikeMcQuaid I looked at pip’s implementation and documentation.
One of the paths (~/.config/pip) can be overridden by the user if they set XDG_CONFIG_HOME. I didn’t take that into account as Homebrew filters that environment variable out anyway (I think).
The other paths are all hard-coded, and I found no obvious method to keep pip from looking at those.
Another way to bend pip to our will is to add --no-user to all places where Homebrew invokes pip install. (That includes e. g. postinstall in python and our poet snippet.)
@Rylan12 and I have discussed and abandoned that alternative. We feel it’s probably not worth the effort.
After that discussion, @sjackman suggested – as a middle ground – what ended up becoming this PR.

Another way to bend pip to our will is to add --no-user to all places where Homebrew invokes pip install. (That includes e. g. postinstall in python and our poet snippet.)
@Rylan12 and I have discussed and abandoned that alternative. We feel it’s probably not worth the effort.

I think that would probably be worth the effort, personally, but I'm happy with this PR as an iterative step in that direction 👍🏻

Another way to bend pip to our will is to add --no-user to all places where Homebrew invokes pip install. (That includes e. g. postinstall in python and our poet snippet.)
@Rylan12 and I have discussed and abandoned that alternative. We feel it’s probably not worth the effort.

I think that would probably be worth the effort, personally, but I'm happy with this PR as an iterative step in that direction 👍🏻

@MikeMcQuaid There’s one more path which pip looks at<EMAIL_ADDRESS>I didn’t include that path in the diagnostics because it’s inside the keg, so users are unlikely to create it.
We may be able to put our own anti-setting there. It will silently override the user-wide setting so I don’t recommend that method for UX reasons.

@MikeMcQuaid There’s one more path which pip looks at<EMAIL_ADDRESS>I didn’t include that path in the diagnostics because it’s inside the keg, so users are unlikely to create it.
We may be able to put our own anti-setting there. It will silently override the user-wide setting so I don’t recommend that method for UX reasons.

I could have sworn I left a comment here last night but it doesn't seem like it made it... Maybe it's sitting in another PR... Anyway, here's essentially what I was saying:
I don't think we should forbid users from configuring their pip. I'm not a power pip user, but I'm sure there are very legitimate uses for using one of these pip configuration files.
Instead, we should adjust our scripts to use the --isolated flag. From pip --help:
  --isolated                  Run pip in an isolated mode, ignoring
                              environment variables and user configuration.

That way, we can have total control over the commands that Homebrew runs while also allowing users to have configurations.
I think this is a slightly different situation than what we agrees on in the docs because these commands are hidden within brew internals, so it's much harder for a user to diagnose. When we're suggesting commands for Homebrew contributors to run, they are more likely to troubleshoot errors on their own so it's less necessary. However, I wouldn't be opposed to including it in the documentation as well.

I could have sworn I left a comment here last night but it doesn't seem like it made it... Maybe it's sitting in another PR... Anyway, here's essentially what I was saying:
I don't think we should forbid users from configuring their pip. I'm not a power pip user, but I'm sure there are very legitimate uses for using one of these pip configuration files.
Instead, we should adjust our scripts to use the --isolated flag. From pip --help:
  --isolated                  Run pip in an isolated mode, ignoring
                              environment variables and user configuration.

That way, we can have total control over the commands that Homebrew runs while also allowing users to have configurations.
I think this is a slightly different situation than what we agrees on in the docs because these commands are hidden within brew internals, so it's much harder for a user to diagnose. When we're suggesting commands for Homebrew contributors to run, they are more likely to troubleshoot errors on their own so it's less necessary. However, I wouldn't be opposed to including it in the documentation as well.

I think this is a slightly different situation than what we agrees on in the docs because these commands are hidden within brew internals

Fair enough.
To paraphrase: we could edit postinstall in our<EMAIL_ADDRESS>formulae to add the --isolated switch, abandon this PR and leave the poet document as is.

I think this is a slightly different situation than what we agrees on in the docs because these commands are hidden within brew internals

Fair enough.
To paraphrase: we could edit postinstall in our<EMAIL_ADDRESS>formulae to add the --isolated switch, abandon this PR and leave the poet document as is.

I'd prefer adding --isolated to the postinstall as well, or maybe even just --no-user if that's the only flag that interferes. Recommending that a user remove their entire pip.conf seems heavy-handed.

I'd prefer adding --isolated to the postinstall as well, or maybe even just --no-user if that's the only flag that interferes. Recommending that a user remove their entire pip.conf seems heavy-handed.

We may also want/need to add to any pip install commands in the Python module. And @dtrodrigues this might be something we want to add to the python -m venv migration effort as well.
Sorry, thanks for doing the work here, @claui ❤

We may also want/need to add to any pip install commands in the Python module. And @dtrodrigues this might be something we want to add to the python -m venv migration effort as well.
Sorry, thanks for doing the work here, @claui ❤

Here's the method I was talking about in the Python module:
https://github.com/Homebrew/brew/blob/30d5fc628574f8330e8acfc2a08d36456fa1b0ff/Library/Homebrew/language/python.rb#L73-L90

Here's the method I was talking about in the Python module:
https://github.com/Homebrew/brew/blob/30d5fc628574f8330e8acfc2a08d36456fa1b0ff/Library/Homebrew/language/python.rb#L73-L90

Sorry, thanks for doing the work here, @claui

No worries. This PR helped shape the new plan so imho the PR has served its purpose.

Sorry, thanks for doing the work here, @claui

No worries. This PR helped shape the new plan so imho the PR has served its purpose.

Here's the method I was talking about in the Python module:
https://github.com/Homebrew/brew/blob/30d5fc628574f8330e8acfc2a08d36456fa1b0ff/Library/Homebrew/language/python.rb#L73-L90

If I read this snippet correctly, it’s an invocation of setup.py, not pip install, right?
Note that this already says --no-user-cfg. Is that setup.py’s equivalent to --isolate?
(I’m relatively new to Python so thanks to y’all for your patience.)

Here's the method I was talking about in the Python module:
https://github.com/Homebrew/brew/blob/30d5fc628574f8330e8acfc2a08d36456fa1b0ff/Library/Homebrew/language/python.rb#L73-L90

If I read this snippet correctly, it’s an invocation of setup.py, not pip install, right?
Note that this already says --no-user-cfg. Is that setup.py’s equivalent to --isolate?
(I’m relatively new to Python so thanks to y’all for your patience.)

setup_install_arg actually can possibly be removed when switching to venv -- it's only used to bootstrap virtualenv.
That being said, adding --isolated to https://github.com/Homebrew/brew/blob/30d5fc628574f8330e8acfc2a08d36456fa1b0ff/Library/Homebrew/language/python.rb#L333-L338
would be a good idea in https://github.com/Homebrew/brew/pull/10039

setup_install_arg actually can possibly be removed when switching to venv -- it's only used to bootstrap virtualenv.
That being said, adding --isolated to https://github.com/Homebrew/brew/blob/30d5fc628574f8330e8acfc2a08d36456fa1b0ff/Library/Homebrew/language/python.rb#L333-L338
would be a good idea in https://github.com/Homebrew/brew/pull/10039

Oops, that's what I meant to paste... 🤦

Oops, that's what I meant to paste... 🤦


Metadata sniffs

Warning:
We need sniffs that check metadata.  Currently, the theme review team allows themes to have metadata if it is presentational in nature.  Some common examples:

Sidebar position
Background color
Layout

However, we do not allow metadata that is non-presentational in nature.  Some common examples:

Video URL
Gallery shortcode
User-specific social URLs

This would probably need to be an warning-level notice because we wouldn't be able to know whether the meta is presentational or not via code.
Code would need to check functions:

add_metadata()
update_metadata()
add_post_meta()
update_post_meta()
add_user_meta()
update_user_meta()
add_comment_meta()
update_comment_meta()

We could probably check for the delete_*_meta() functions too, but this is probably unnecessary.
Hooks to check:

attachment_fields_to_edit
attachment_fields_to_save

Error:
One area we could absolutely check for non-presentational metadata is if a theme has a filter on user_contactmethods.  This is only used for adding contact methods to the user profile, which would not be presentational.

And how would this work if the theme integrates a library for that purpose? Is it allowed because the theme isn't doing the work? Or is the library then part of the theme that shouldn't be doing it either?

A library included in the theme is still a part of the theme.  So, it doesn't matter.  The library is irrelevant in this case.

I suppose we should do the same for custom widgets.

Let allow non-presentational metadata.

There is a branch for this issue-136-non-presentational-meta but it needs testing.
Should this not be combined with other restricted functions?

Resolution from the triage: we should add this as a sniff and add info into the requirements. Also, see if there is a chapter for this in the Theme developers handbook and add a chapter there if there isn't


Is there a way to get all the available routes?

With dev mode, when you go to a 404 page, you get a list of all the routes, is there a way that we can also get all the routes without going to that page, but rather through SSR?

This looks like a Help question. Could you ask in the Discord qwik-help channel instead?
Although, just in case, maybe you're looking for menus?

This looks like a Help question. Could you ask in the Discord qwik-help channel instead and close the issue? You can ping me @maiieul
Although, just in case, maybe you're looking for menus?

Not looking for menus, I am looking for a way to check if the route is an actual route for the breadcrumb component since sometimes the route is not available and I have it as a link, but I don't want to manually enter all the routes I have, though..

import qwikCityPlan from '@qwik-city-plan'

export const onGet: RequestHandler = () => {
  // Not sure what the types are but they arent properly exposed
  const routes = qwikCityPlan.routes as [any, any, any, string, any][]

  for (const route of routes) {
    console.log(route[3])
  }
}

import qwikCityPlan from '@qwik-city-plan'

export const onGet: RequestHandler = () => {
  // Not sure what the types are but they arent properly exposed
  const routes = qwikCityPlan.routes as [any, any, any, string, any][]

  for (const route of routes) {
    console.log(route[3])
  }
}

You're a freaking legend, man! Thanks a lot!


Multiple template dirs support for deploy task

What are you trying to accomplish with this PR?

Implement support for passing --template-dir flag multiple times

Must support proper partials scoping
Must work with templates passed from stdin (tbh this is mostly for free)

Currently this is only for deploy_task
Required some changes to method signatures in the tests. I tried to make these as minor as possible.

How is this accomplished?
Initially, I tried passing multiple template directories around deploy_task, but this became cumbersome: I had to create a mapping of template directories to their respective partials dirs, wrap them together, and pass them to and from TemplateContext objects and back to the (somewhat-global) renderer.
A cleaner approach was to simply instantiate a separate renderer for each template directory passed in, then, when appropriate, concatenate the result of each template directory rendering operation and proceed through the deploy as normal. This way, the changes to the codebase are much smaller and the logic for each individual element stays simpler.
Edit: What I ended up doing in the end is a bit of a mix of the first 2 ideas. In the deploy task constructor, we declare a mapping of renderers to template directories. The renderers are then lazily instantiated directly at the point of rendering (in #split_templates), leveraging the closure provided by the default block of the @renderers Hash (thank you Blake for the great suggestion).
Please see the unresolved comments from my review for more fine-grained issues regarding this PR.
What could go wrong?
How defensive should we be? I think that giving people the ability to specify multiple template directories places the responsibility of ensuring they end up with a sane and collision-free experience squarely on them.
TODO

[x] Add tests

My original concern with this PR was how will it interact with our goal of supporting a -f flag in the Krane CLI https://docs.google.com/document/d/1oInUsKplYGNWTymPY48xtDCx3x4Cwc-phcEKUkt2v4U/edit#. I think this actually makes it easier @timothysmith0609 thoughts?

My original concern with this PR was how will it interact with our goal of supporting a -f flag in the Krane CLI https://docs.google.com/document/d/1oInUsKplYGNWTymPY48xtDCx3x4Cwc-phcEKUkt2v4U/edit#. I think this actually makes it easier @timothysmith0609 thoughts?

At the very least, I don't think this PR makes things any more difficult in achieving that end. The sole complicating factor between kubectl -f and krane deploy -f is our support of partials. I think there's a few ways we can tackle that:

When given a directory/directories, proceed as we currently do (this PR actually enables this behaviour)
When given concrete filenames (not directories), collect them together by directory, and treat those collections as more template_dirs (this will require more thought than just this, of course).

A cleaner approach was to simply instantiate a separate renderer for each template directory passed in, then, when appropriate, concatenate the result of each template directory rendering operation and proceed through the deploy as normal. This way, the changes to the codebase are much smaller and the logic for each individual element stays simpler.
Edit: What I ended up doing in the end is a bit of a mix of the first 2 ideas.

Can you comment on why the cleaner approach didn't work out?

Can you comment on why the cleaner approach didn't work out?

The renderer takes a few arguments that are easily grabbed in the task initializer. In the end, it seemed easier to wrap those in the closure provided by the default hash block than try to pass them into TemplateDiscovery further down the stack. That's really the only difference

Perhaps TemplateDiscovery is poorly named, and should instead be something like ResourceDiscovery. ResourceDiscovery is then responsible for finding all relevant resource files in a template directories, splitting the templates in each file, calling the renderer on them, and returning a list of fully realized KubernetesResources.

I can't comment in the thread but, I like the idea. As an fyi we have a class called ClusterResourceDiscovery so I'd prefer something more specific  LocalResouceDiscovery?


Update functions-run-local.md

Learn Build status updates of commit 53ee0b2:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

articles/azure-functions/functions-run-local.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

@ggailey777
Can you review the proposed changes? IMPORTANT: When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

Thanks @vamshi9666 for the fix. Sorry it took so long to get to: #sign-off


proposal: add a take_until_parser which return consume input until a parser success

here is the reason, when parsing commonmark's code block, it should be delimited by the same length of ` or ~, where we need to keep track of the length of beginning `, and might need to omit ending space. I found it's hard to just use nom to deal with it.
In another parser combinator I used before called sprache(C#) has a Except method to do that thing, I think it might be useful.

I've implement one in my repo

I've implement one in my repo


Rename com.timgroup package

This library is effectively repackaging the library it was forked from. This causes classpath loading issues.
If you have a library that depends on the forked library and another one which depends on this, you end up with two classes with the same name with possibly different binary interfaces (e.g. in the count method on NonBlockingStatsDClient).
Please never ever do something foolish as this. Just properly rename that package.

why would anybody depend on both libraries?

Transitive dependencies. If A depends on B which depends on the original statsd and A depends on C which depends on dogstatsd you end up having two dependencies on your classpath exporting the same package.

I don't know how common the scenario you described is. If I changed the package name, I would be breaking backwards compatibility.
Also, I don't think that every time someone forks a java repo they also change the package name. I'll ask a few colleagues and come back to you with a more definite answer.

I agree with aranbk's answer here.  Closing this.


[Improvement][Dao] Remove duplicated mysql configurations

Tips

Thanks very much for contributing to Apache DolphinScheduler.
Please review https://dolphinscheduler.apache.org/en-us/community/index.html before opening a pull request.

What is the purpose of the pull request
Remove duplicated mysql configurations
Brief change log

Remove duplicated mysql configurations
Sync .gitignore with the 1.3.5-prepare branch for merging the next pr and avoiding conflicts

Verify this pull request
This change added tests and can be verified as follows:
(example:)

Manually verified the change by testing locally.

Codecov Report

Merging #4558 (33c6d05) into dev (69445e3) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##                dev    #4558   +/-   ##
=========================================
  Coverage     44.71%   44.71%
  Complexity     3340     3340
=========================================
  Files           530      530
  Lines         22838    22838
  Branches       2669     2669
=========================================
  Hits          10213    10213
  Misses        11704    11704
  Partials        921      921

Impacted Files
Coverage Δ
Complexity Δ

...er/master/dispatch/host/assign/RandomSelector.java
77.77% <0.00%> (-5.56%)
3.00% <0.00%> (-1.00%)

...dolphinscheduler/remote/future/ResponseFuture.java
83.05% <0.00%> (+1.69%)
19.00% <0.00%> (+1.00%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 69445e3...33c6d05. Read the comment docs.

Codecov Report

Merging #4558 (33c6d05) into dev (69445e3) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##                dev    #4558   +/-   ##
=========================================
  Coverage     44.71%   44.71%
  Complexity     3340     3340
=========================================
  Files           530      530
  Lines         22838    22838
  Branches       2669     2669
=========================================
  Hits          10213    10213
  Misses        11704    11704
  Partials        921      921

Impacted Files
Coverage Δ
Complexity Δ

...er/master/dispatch/host/assign/RandomSelector.java
77.77% <0.00%> (-5.56%)
3.00% <0.00%> (-1.00%)

...dolphinscheduler/remote/future/ResponseFuture.java
83.05% <0.00%> (+1.69%)
19.00% <0.00%> (+1.00%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 69445e3...33c6d05. Read the comment docs.


Add changesets for DAM scoping (PR #976)

Add changesets for DAM scoping (PR #976)

@thomasdax98 I can't set you as reviewer since you created the PR... @nsams could you please review?


Failure to compile the project (Version <IP_ADDRESS>) with Visual Studio

Hello,
I have downloaded MahApps.Metro.<IP_ADDRESS>.nupkg and Source code (zip).
I am using Windows 7 and Visual Studio 2013.
.NET Framework 4.0, 4.5, 4.5.1, 4.5.2, 4.6 and 4.6.1 are installed.
I have opened the project from the ZIP and install the package (nupkg) without problem.
When I try to build the project, I get 68 errors and 1 warning (The 'packages' element is not declared).
I get that type of error during compilation:
A) ValidationResult(false, $"The price {album.Price} of the album '{album.Title}' by '{album.Artist.Name}' is too much!") ==> Unexpected character '$' for the error (class AlbumPriceIsTooMuchConverter)
B) bindingGroup?.Items.OfType().ElementAtOrDefault(0);  ==> Invalid expression term '.'
C) internal static KeyboardNavigationEx Instance => _instance ?? (_instance = new KeyboardNavigationEx()); ==> Invalid token '??' in class, struct, or interface member declaration (class KeyboardNavigationEx)
Could you help / guide me so I can compile and make run the project please.
PS: Files were downloaded from here: https://github.com/MahApps/MahApps.Metro/releases/tag/1.3.0

@boukenka You need VS 2015 if you want to use C# 6 features. You can take the Community edition -> it's free

Thanks for the answer. I have installed VS2015 and install the package.
But now I am getting other type of errors.
Error	CS0121	The call is ambiguous between the following methods or properties: 'MahApps.Metro.Controls.Extensions.BeginInvoke(System.Windows.Threading.DispatcherObject, System.Action, System.Windows.Threading.DispatcherPriority)' and 'MahApps.Metro.Controls.Extensions.BeginInvoke(System.Windows.Threading.DispatcherObject, System.Action, System.Windows.Threading.DispatcherPriority)'	MahApps.Metro.NET45

I have a total of 265 errors and 1615 warnings. Could you please let me know how to solve it please.

Hello, Is there someone who could answer to me?

@boukenka It looks like you referenced the compiled dll and the source project in your solution. Make sure you do only one of them.

Thanks. It is solving the issue.
I have just a remark. When the 'MahApps.Metro.<IP_ADDRESS>.nupkg' package is applied, it creates reference of 'MahApps.Metro' in the 'MahApps.Metro.NET45' project.

@boukenka You don't need the Nuget package if you compile the source of MahApps.


Model's run_started_at is incrementing even though it isn't part of the selection

I think this one is more of a clarifying question.

Blow away all tables/views in the schema we're going to reproduce this in so as to not have any lingering dbt_artifacts tables/views.

Setup project:

# dbt_project.yml
name: my_dbt_project
profile: snowflake
config-version: 2
version: 1.0

models:
  my_dbt_project:
    +materialized: table

on-run-end: "{{ dbt_artifacts.upload_results(results) }}"

# packages.yml
packages:
  - package: brooklyn-data/dbt_artifacts
    version: 2.2.2

-- models/foo.sql
select 1 as id

-- models/bar.sql
select 1 as id

Run project:

$ dbt clean && dbt deps && dbt run
...
03:36:32  Running with dbt=1.3.2
03:36:34  Found 36 models, 0 tests, 0 snapshots, 0 analyses, 367 macros, 1 operation, 0 seed files, 0 sources, 0 exposures, 0 metrics
03:36:34
03:36:43  Concurrency: 8 threads (target='dev')
03:36:43
03:36:43  1 of 36 START sql incremental model dbt_jyeo.exposures ......................... [RUN]
03:36:43  2 of 36 START sql incremental model dbt_jyeo.invocations ....................... [RUN]
03:36:43  3 of 36 START sql incremental model dbt_jyeo.model_executions .................. [RUN]
03:36:43  4 of 36 START sql incremental model dbt_jyeo.models ............................ [RUN]
03:36:43  5 of 36 START sql incremental model dbt_jyeo.seed_executions ................... [RUN]
03:36:43  6 of 36 START sql incremental model dbt_jyeo.seeds ............................. [RUN]
03:36:43  7 of 36 START sql incremental model dbt_jyeo.snapshot_executions ............... [RUN]
03:36:43  8 of 36 START sql incremental model dbt_jyeo.snapshots ......................... [RUN]
03:36:48  3 of 36 OK created sql incremental model dbt_jyeo.model_executions ............. [SUCCESS 1 in 4.31s]
03:36:48  9 of 36 START sql incremental model dbt_jyeo.sources ........................... [RUN]
03:36:48  5 of 36 OK created sql incremental model dbt_jyeo.seed_executions .............. [SUCCESS 1 in 5.03s]
03:36:48  10 of 36 START sql incremental model dbt_jyeo.test_executions .................. [RUN]
03:36:48  6 of 36 OK created sql incremental model dbt_jyeo.seeds ........................ [SUCCESS 1 in 5.26s]
03:36:49  11 of 36 START sql incremental model dbt_jyeo.tests ............................ [RUN]
03:36:49  1 of 36 OK created sql incremental model dbt_jyeo.exposures .................... [SUCCESS 1 in 5.31s]
03:36:49  12 of 36 START sql table model dbt_jyeo.bar .................................... [RUN]
03:36:49  7 of 36 OK created sql incremental model dbt_jyeo.snapshot_executions .......... [SUCCESS 1 in 5.47s]
03:36:49  13 of 36 START sql table model dbt_jyeo.foo .................................... [RUN]
03:36:49  4 of 36 OK created sql incremental model dbt_jyeo.models ....................... [SUCCESS 1 in 5.60s]
03:36:49  14 of 36 START sql view model dbt_jyeo.stg_dbt__model_executions ............... [RUN]
03:36:49  8 of 36 OK created sql incremental model dbt_jyeo.snapshots .................... [SUCCESS 1 in 6.17s]
03:36:49  15 of 36 START sql view model dbt_jyeo.stg_dbt__seed_executions ................ [RUN]
03:36:49  2 of 36 OK created sql incremental model dbt_jyeo.invocations .................. [SUCCESS 1 in 6.20s]
03:36:49  16 of 36 START sql view model dbt_jyeo.stg_dbt__seeds .......................... [RUN]
03:36:52  9 of 36 OK created sql incremental model dbt_jyeo.sources ...................... [SUCCESS 1 in 4.24s]
03:36:52  17 of 36 START sql view model dbt_jyeo.stg_dbt__exposures ...................... [RUN]
03:36:52  16 of 36 OK created sql view model dbt_jyeo.stg_dbt__seeds ..................... [SUCCESS 1 in 2.98s]
03:36:52  18 of 36 START sql view model dbt_jyeo.stg_dbt__snapshot_executions ............ [RUN]
03:36:52  14 of 36 OK created sql view model dbt_jyeo.stg_dbt__model_executions .......... [SUCCESS 1 in 3.59s]
03:36:52  19 of 36 START sql view model dbt_jyeo.stg_dbt__models ......................... [RUN]
03:36:52  15 of 36 OK created sql view model dbt_jyeo.stg_dbt__seed_executions ........... [SUCCESS 1 in 3.05s]
03:36:52  20 of 36 START sql view model dbt_jyeo.stg_dbt__snapshots ...................... [RUN]
03:36:53  10 of 36 OK created sql incremental model dbt_jyeo.test_executions ............. [SUCCESS 1 in 4.57s]
03:36:53  21 of 36 START sql view model dbt_jyeo.stg_dbt__invocations .................... [RUN]
03:36:53  12 of 36 OK created sql table model dbt_jyeo.bar ............................... [SUCCESS 1 in 4.19s]
03:36:53  22 of 36 START sql view model dbt_jyeo.stg_dbt__sources ........................ [RUN]
03:36:53  13 of 36 OK created sql table model dbt_jyeo.foo ............................... [SUCCESS 1 in 4.19s]
03:36:53  11 of 36 OK created sql incremental model dbt_jyeo.tests ....................... [SUCCESS 1 in 4.34s]
03:36:53  23 of 36 START sql view model dbt_jyeo.dim_dbt__seeds .......................... [RUN]
03:36:53  24 of 36 START sql view model dbt_jyeo.fct_dbt__model_executions ............... [RUN]
03:36:54  17 of 36 OK created sql view model dbt_jyeo.stg_dbt__exposures ................. [SUCCESS 1 in 2.62s]
03:36:54  25 of 36 START sql view model dbt_jyeo.fct_dbt__seed_executions ................ [RUN]
03:36:56  18 of 36 OK created sql view model dbt_jyeo.stg_dbt__snapshot_executions ....... [SUCCESS 1 in 3.76s]
03:36:56  19 of 36 OK created sql view model dbt_jyeo.stg_dbt__models .................... [SUCCESS 1 in 3.74s]
03:36:56  26 of 36 START sql view model dbt_jyeo.stg_dbt__test_executions ................ [RUN]
03:36:56  27 of 36 START sql view model dbt_jyeo.stg_dbt__tests .......................... [RUN]
03:36:56  20 of 36 OK created sql view model dbt_jyeo.stg_dbt__snapshots ................. [SUCCESS 1 in 3.81s]
03:36:56  28 of 36 START sql view model dbt_jyeo.dim_dbt__exposures ...................... [RUN]
03:36:56  22 of 36 OK created sql view model dbt_jyeo.stg_dbt__sources ................... [SUCCESS 1 in 3.47s]
03:36:56  29 of 36 START sql view model dbt_jyeo.fct_dbt__snapshot_executions ............ [RUN]
03:36:56  21 of 36 OK created sql view model dbt_jyeo.stg_dbt__invocations ............... [SUCCESS 1 in 3.53s]
03:36:56  30 of 36 START sql view model dbt_jyeo.dim_dbt__current_models ................. [RUN]
03:36:57  23 of 36 OK created sql view model dbt_jyeo.dim_dbt__seeds ..................... [SUCCESS 1 in 3.67s]
03:36:57  24 of 36 OK created sql view model dbt_jyeo.fct_dbt__model_executions .......... [SUCCESS 1 in 3.67s]
03:36:57  32 of 36 START sql view model dbt_jyeo.dim_dbt__snapshots ...................... [RUN]
03:36:57  31 of 36 START sql view model dbt_jyeo.dim_dbt__models ......................... [RUN]
03:36:58  25 of 36 OK created sql view model dbt_jyeo.fct_dbt__seed_executions ........... [SUCCESS 1 in 2.57s]
03:36:58  33 of 36 START sql view model dbt_jyeo.dim_dbt__sources ........................ [RUN]
03:36:59  26 of 36 OK created sql view model dbt_jyeo.stg_dbt__test_executions ........... [SUCCESS 1 in 3.31s]
03:36:59  34 of 36 START sql view model dbt_jyeo.fct_dbt__invocations .................... [RUN]
03:37:00  27 of 36 OK created sql view model dbt_jyeo.stg_dbt__tests ..................... [SUCCESS 1 in 3.68s]
03:37:00  35 of 36 START sql view model dbt_jyeo.fct_dbt__test_executions ................ [RUN]
03:37:00  30 of 36 OK created sql view model dbt_jyeo.dim_dbt__current_models ............ [SUCCESS 1 in 3.81s]
03:37:00  36 of 36 START sql view model dbt_jyeo.dim_dbt__tests .......................... [RUN]
03:37:01  29 of 36 OK created sql view model dbt_jyeo.fct_dbt__snapshot_executions ....... [SUCCESS 1 in 4.21s]
03:37:01  28 of 36 OK created sql view model dbt_jyeo.dim_dbt__exposures ................. [SUCCESS 1 in 4.25s]
03:37:01  31 of 36 OK created sql view model dbt_jyeo.dim_dbt__models .................... [SUCCESS 1 in 4.21s]
03:37:01  32 of 36 OK created sql view model dbt_jyeo.dim_dbt__snapshots ................. [SUCCESS 1 in 4.22s]
03:37:01  33 of 36 OK created sql view model dbt_jyeo.dim_dbt__sources ................... [SUCCESS 1 in 2.78s]
03:37:02  34 of 36 OK created sql view model dbt_jyeo.fct_dbt__invocations ............... [SUCCESS 1 in 2.53s]
03:37:02  35 of 36 OK created sql view model dbt_jyeo.fct_dbt__test_executions ........... [SUCCESS 1 in 2.53s]
03:37:03  36 of 36 OK created sql view model dbt_jyeo.dim_dbt__tests ..................... [SUCCESS 1 in 2.85s]
03:37:03
03:37:03  Running 1 on-run-end hook
03:37:03  Uploading model executions
03:37:05  Uploading seed executions
03:37:05  Uploading snapshot executions
03:37:05  Uploading test executions
03:37:05  Uploading exposures
03:37:05  Uploading tests
03:37:05  Uploading seeds
03:37:05  Uploading models
03:37:06  Uploading sources
03:37:06  Uploading snapshots
03:37:06  Uploading invocations
03:37:07  1 of 1 START hook: my_dbt_project.on-run-end.0 ................................. [RUN]
03:37:07  1 of 1 OK hook: my_dbt_project.on-run-end.0 .................................... [OK in 0.00s]
03:37:07
03:37:07
03:37:07  Finished running 11 incremental models, 23 view models, 2 table models, 1 hook in 0 hours 0 minutes and 33.36 seconds (33.36s).
03:37:07
03:37:07  Completed successfully
03:37:07
03:37:07  Done. PASS=36 WARN=0 ERROR=0 SKIP=0 TOTAL=36

Inspect dbt_artifacts tables:

Run again but select only one of the models:

$ dbt run -s bar
03:39:21  Running with dbt=1.3.2
03:39:23  Found 36 models, 0 tests, 0 snapshots, 0 analyses, 367 macros, 1 operation, 0 seed files, 0 sources, 0 exposures, 0 metrics
03:39:23
03:39:29  Concurrency: 8 threads (target='dev')
03:39:29
03:39:29  1 of 1 START sql table model dbt_jyeo.bar ...................................... [RUN]
03:39:32  1 of 1 OK created sql table model dbt_jyeo.bar ................................. [SUCCESS 1 in 3.21s]
03:39:32
03:39:32  Running 1 on-run-end hook
03:39:32  Uploading model executions
03:39:35  Uploading seed executions
03:39:35  Uploading snapshot executions
03:39:35  Uploading test executions
03:39:35  Uploading exposures
03:39:35  Uploading tests
03:39:35  Uploading seeds
03:39:35  Uploading models
03:39:36  Uploading sources
03:39:36  Uploading snapshots
03:39:36  Uploading invocations
03:39:37  1 of 1 START hook: my_dbt_project.on-run-end.0 ................................. [RUN]
03:39:37  1 of 1 OK hook: my_dbt_project.on-run-end.0 .................................... [OK in 0.00s]
03:39:37
03:39:37
03:39:37  Finished running 1 table model, 1 hook in 0 hours 0 minutes and 14.09 seconds (14.09s).
03:39:37
03:39:37  Completed successfully
03:39:37
03:39:37  Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1

Inspect dbt_artifacts tables again:

^ And so it appears that foo was also logged in dbt_artifacts as being part of the run for some reason.
Should this be how dbt_artifacts works? Model foo - not part of the second run - was also logged as being part of the run in the various dbt_artifacts tables. My mental model of how this should work is that foo should not have an additional line added here.
Thanks :)

Actually - if we inspect the debug logs - we get some hints - for my second run with only bar selected, we're inserting into models table foo as well...
03:39:35.447376 [debug] [MainThread]: On master: /* {"app": "dbt", "dbt_version": "1.3.2", "profile_name": "snowflake", "target_name": "dev", "connection_name": "master"} */
insert into development_jyeo.dbt_jyeo.models

        select
            $1,
            $2,
            $3,
            $4,
            $5,
            $6,
            parse_json($7),
            $8,
            $9,
            $10,
            $11,
            parse_json($12),
            parse_json($13),
            $14
        from values
        (
                '29e729b2-ffad-484c-89e8-5a1ef424905e',
                'model.my_dbt_project.foo',
                '2023-02-09 03:39:19.969571+00:00',
                'development_jyeo',
                'dbt_jyeo',
                'foo',
                '[]',
                'my_dbt_project',
                'models/foo.sql',
                '479636cb85ce8d3b0f8db5ff13cf338b61254ad98d905630eac61f963e719e9d',
                'table',
                '[]',
                '{}',
                'foo'
            ),(
                '29e729b2-ffad-484c-89e8-5a1ef424905e',
                'model.my_dbt_project.bar',
                '2023-02-09 03:39:19.969571+00:00',
                'development_jyeo',
                'dbt_jyeo',
                'bar',
                '[]',
                'my_dbt_project',
                'models/bar.sql',
                '479636cb85ce8d3b0f8db5ff13cf338b61254ad98d905630eac61f963e719e9d',
                'table',
                '[]',
                '{}',
                'bar'
            ),(

...

Okay so it appears that I missed out a fine print:

If you are using selectors, be sure to include the dbt_artifacts models in your dbt invocation step.

So let's try the above again take 2 and see the outcome.
$ dbt run
00:44:43  Running with dbt=1.3.2
00:44:45  Found 36 models, 0 tests, 0 snapshots, 0 analyses, 367 macros, 1 operation, 0 seed files, 0 sources, 0 exposures, 0 metrics
00:44:45
00:44:57  Concurrency: 8 threads (target='dev')
00:44:57
00:44:57  1 of 36 START sql incremental model dbt_jyeo.exposures ......................... [RUN]
00:44:57  2 of 36 START sql incremental model dbt_jyeo.invocations ....................... [RUN]
00:44:57  3 of 36 START sql incremental model dbt_jyeo.model_executions .................. [RUN]
00:44:57  4 of 36 START sql incremental model dbt_jyeo.models ............................ [RUN]
00:44:57  5 of 36 START sql incremental model dbt_jyeo.seed_executions ................... [RUN]
00:44:57  6 of 36 START sql incremental model dbt_jyeo.seeds ............................. [RUN]
00:44:57  7 of 36 START sql incremental model dbt_jyeo.snapshot_executions ............... [RUN]
00:44:57  8 of 36 START sql incremental model dbt_jyeo.snapshots ......................... [RUN]
00:45:02  1 of 36 OK created sql incremental model dbt_jyeo.exposures .................... [SUCCESS 1 in 4.81s]
00:45:02  9 of 36 START sql incremental model dbt_jyeo.sources ........................... [RUN]
00:45:02  6 of 36 OK created sql incremental model dbt_jyeo.seeds ........................ [SUCCESS 1 in 5.25s]
...
00:45:19  Uploading snapshots
00:45:19  Uploading invocations
00:45:20  1 of 1 START hook: my_dbt_project.on-run-end.0 ................................. [RUN]
00:45:20  1 of 1 OK hook: my_dbt_project.on-run-end.0 .................................... [OK in 0.00s]
00:45:20
00:45:20
00:45:20  Finished running 11 incremental models, 23 view models, 2 table models, 1 hook in 0 hours 0 minutes and 34.75 seconds (34.75s).
00:45:20
00:45:20  Completed successfully
00:45:20
00:45:20  Done. PASS=36 WARN=0 ERROR=0 SKIP=0 TOTAL=36

$ dbt run -s bar dbt_artifacts
00:47:54  Running with dbt=1.3.2
00:47:56  Found 36 models, 0 tests, 0 snapshots, 0 analyses, 367 macros, 1 operation, 0 seed files, 0 sources, 0 exposures, 0 metrics
00:47:56
00:48:03  Concurrency: 8 threads (target='dev')
00:48:03
00:48:03  1 of 35 START sql incremental model dbt_jyeo.exposures ......................... [RUN]
00:48:03  2 of 35 START sql incremental model dbt_jyeo.invocations ....................... [RUN]
00:48:03  3 of 35 START sql incremental model dbt_jyeo.model_executions .................. [RUN]
00:48:03  4 of 35 START sql incremental model dbt_jyeo.models ............................ [RUN]
...
00:48:28  Uploading tests
00:48:28  Uploading seeds
00:48:29  Uploading models
00:48:29  Uploading sources
00:48:29  Uploading snapshots
00:48:29  Uploading invocations
00:48:30  1 of 1 START hook: my_dbt_project.on-run-end.0 ................................. [RUN]
00:48:30  1 of 1 OK hook: my_dbt_project.on-run-end.0 .................................... [OK in 0.00s]
00:48:30
00:48:31
00:48:31  Finished running 11 incremental models, 23 view models, 1 table model, 1 hook in 0 hours 0 minutes and 34.92 seconds (34.92s).
00:48:31
00:48:31  Completed successfully
00:48:31
00:48:31  Done. PASS=35 WARN=0 ERROR=0 SKIP=0 TOTAL=35

From what I can tell - the behaviour is consistent as previously (when I did not include dbt_artifacts as part of the selector.

Hi @jeremyyeo 👋  thanks for the detailed write up here and sorry for the delay getting back to you. Based on my reading of the issue what you're describing is the code working as written.
upload_models relies on the graph object whereas upload_model_executions relies on the results object. Since dbt's graph is complete regardless of which sub graph you select on a given invocation all of the downstream dbt_artficats models are going to upload the graph in it's entirety and use the run_started_at date from that invocation even if the model is not selected.
From the docs on run_started_at

The start timestamp of the dbt execution which generated the record.

Is it fair to say that you would expect the behaviour to be such that current_models would have the run_started_at timestamp properly inherited from fct_dbt__model_executions?

It looks like this question has now been resolved, so I'm going to close this issue. Please feel free to reopen it if you do not feel it was appropriately answered.


build: .in files should be regenerated after a .config change

If we don't regenerate the .in files they will end up using the wrong
prefix if it has been reset.
Signed-off-by: Leandro Dorileo<EMAIL_ADDRESS>

+1

applied.


getUsername does not resolve referenced field's value

We are using KeePass for tracking service account credentials per application in our environment. It is possible, we will have the same user/pass for multiple applications--but our system requires a unique KeePass entry. As a result, we've tried out adding new entries that have the Username and Password fields referenence an existing/master entry. In the KeePass UI, you'll see the value for the stored Username (or Password or whatever field you're adding reference too) is something in the form of {REF:U@I:36F89B594FD7664E89E62E8CE220129A}. Once we started doing that, our automation broke (that is using the OpenKeePass library) because the usernames & passwords returned weren't the linked values but the links themselves, ie: {REF:U@I:36F89B594FD7664E89E62E8CE220129A}.
I've done my best to dig into the code and don't see any support for this concept. Is it possible to add the ability to resolve the value if an Entry has a linked reference?
Example:
Title: Item 1, Username: masterUser, UUID 36F89B594FD7664E89E62E8CE220129A
Title: Item 2, Username: {REF:U@I:36F89B594FD7664E89E62E8CE220129A}
When doing database.getEntryByTitle("Item 2").getUsername(); it would be nice to have masterUser returned.
Thanks!

Interesting! I was not even aware that this feature exists :) I need to look into the concept to see how it should work. Depending on the required effort I will decide if I will support it.