variable kerning doesn't seem to activate

After https://github.com/googlefonts/fontc/pull/373 I seem to be seeing that variable kerning doesn't actually activate. For example:
$ export RUST_LOG=info,fontbe=trace
$ rm -rf build/ && cargo run -p fontc -- --source resources/testdata/glyphs3/WghtVar.glyphs
...noise
# fontc generated kerning

@public.kern1.bracketright_R = [bracketright];
@public.kern1.bracketleft_R = [bracketleft];
@public.kern2.bracketright_L = [bracketright];
@public.kern2.bracketleft_L = [bracketleft];

feature kern {
  pos exclam @public.kern2.bracketright_L (wght=400:-160 wght=700:0);
  pos exclam exclam (wght=400:-360 wght=700:-100);
  pos hyphen hyphen (wght=400:-150 wght=700:-50);
  pos bracketleft bracketright (wght=400:-300 wght=700:-150);
  pos @public.kern1.bracketleft_R exclam (wght=400:-165 wght=700:0);
  pos exclam hyphen (wght=400:20 wght=700:0);
} kern;
...

# OK, great, [] should kern -300 at wght=400, -150 at wght=700 per pos bracketleft bracketright (wght=400:-300 wght=700:-150);
# [] have width=600
$  ../harfbuzz/build/util/hb-shape build/font.ttf --text='[' --no-clusters
[bracketleft+600]
$  ../harfbuzz/build/util/hb-shape build/font.ttf --text=']' --no-clusters
[bracketright+600]

# 400 seems to work, bracketleft isn't 600 wide
$ ../harfbuzz/build/util/hb-shape build/font.ttf --text='[]' --no-clusters
[bracketleft+300|bracketright+600]

# but if we push toward 700 bracketleft doesn't change width, it just sits at 300
$ ../harfbuzz/build/util/hb-shape build/font.ttf --text='[]' --no-clusters --variations="wght=500"
[bracketleft+300|bracketright+600]
$ ../harfbuzz/build/util/hb-shape build/font.ttf --text='[]' --no-clusters --variations="wght=700"
[bracketleft+300|bracketright+600]

So there are a few weird things going on here, but a major one is that we aren't building the GDEF table, and that's where the variation data is stored.

so are we just throwing away the ItemVariationStore that write-fonts built?

somewhere here we should store the GDEF we get from fea-rs, like we do for GPOS/GSUB:
https://github.com/googlefonts/fontc/blob/a4ee74ce44df64ccdcd76476f8cc3076d9b0b4c6/fontbe/src/features.rs#L147-L165

With #405 merged, we now have a GDEF with an ItemVariationStore that is doing something, even though not exactly correct yet:
# the advance width of bracketleft should be 450 (600 - 150) at wght=700, but we get 150...
$ hb-shape build/font.ttf --text='[]' --no-clusters --variations="wght=700"
[bracketleft+150|bracketright+600]

the GDEF table that we get from fea-rs for the resources/testdata/glyphs3/WghtVar.glyphs looks like this:
  <GDEF>
    <Version value="0x00010003"/>
    <VarStore Format="1">
      <Format value="1"/>
      <VarRegionList>
        <!-- RegionAxisCount=1 -->
        <!-- RegionCount=2 -->
        <Region index="0">
          <VarRegionAxis index="0">
            <StartCoord value="0.0"/>
            <PeakCoord value="0.0"/>
            <EndCoord value="0.0"/>
          </VarRegionAxis>
        </Region>
        <Region index="1">
          <VarRegionAxis index="0">
            <StartCoord value="0.0"/>
            <PeakCoord value="1.0"/>
            <EndCoord value="1.0"/>
          </VarRegionAxis>
        </Region>
      </VarRegionList>
      <!-- VarDataCount=1 -->
      <VarData index="0">
        <!-- ItemCount=6 -->
        <NumShorts value="2"/>
        <!-- VarRegionCount=2 -->
        <VarRegionIndex index="0" value="0"/>
        <VarRegionIndex index="1" value="1"/>
        <Item index="0" value="[20, -20]"/>
        <Item index="1" value="[-150, 100]"/>
        <Item index="2" value="[-300, 150]"/>
        <Item index="3" value="[-160, 160]"/>
        <Item index="4" value="[-165, 165]"/>
        <Item index="5" value="[-360, 260]"/>
      </VarData>
    </VarStore>
  </GDEF>

There's an extra region at index=0 of VarRegionList, with wght=(min=0, peak=0, max=0) which looks like it should not be there, not sure yet how it ended up there

found the culprit and fixed in 632db7a
Finally we have variable kerning! 🎉

Finally we have variable kerning!

ok well.. at least we do for the simple WghtVar.glyphs test font...
I tested Oswald as well, comparing it with the fontmake-built font, and I am still seeing some pairs that do not get any kerning at all. But this happens regardless of variations, even at the default instance I am seeing differences.
E.g. "Va" kerning is working but "aV" is not, but it should.. 🤔
$ hb-shape build/Oswald-VF-fontc.ttf --text='VaV' --no-clusters --features=-kern
[V+505|a+426|V+505]
$ hb-shape build/Oswald-VF-fontmake.ttf --text='VaV' --no-clusters --features=-kern
[V+505|a+426|V+505]
$ hb-shape build/Oswald-VF-fontc.ttf --text='VaV' --no-clusters
[V+455|a+426|V+505]
$ hb-shape build/Oswald-VF-fontmake.ttf --text='VaV' --no-clusters
[V+455|a+378|V+505]

I'm suspecting it could have something to do with PairPos subtables being split, cf. https://github.com/googlefonts/fontations/pull/401/ (though I haven't been able to confirm this so far)

ok, I figured the reason was for class to glyph or glyph to class kerning pairs (also known as class kerning "exceptions") such as the offending @a V pair in Oswald, we were also using pos instead of enum pos, so instead of expanding them to specific glyph-glyph pairs, the compiler was using class pairs for these, which leads to conflicting/invalid class memberships. ufo2ft kernFeatureWriter adds enum pos to all enumerated = pair.firstIsClass ^ pair.secondIsClass, see https://github.com/googlefonts/ufo2ft/blob/b3895a96ca910c1764df016bfee4719448cfec4a/Lib/ufo2ft/featureWriters/kernFeatureWriter.py#L360C26-L360C39
I have fixed it in https://github.com/googlefonts/fontc/pull/373/commits/9661684f112dfdd9c4554a2cdf8dd0e5f98ecbde and now I am seeing variable kerning work for Oswald. I have only done a spot visual check, not a deep diff, but results are encouraging.

Variable kerning certainly seems to activate so I think the core issue here is resolved. We can file additional issues as we dig into why fontc and fontmake don't produce the same GPOS.


[Flaky test] ExtendedStatsAggregationUsageDocCountZeroTests returns null values

ExtendedStatsAggregationUsageDocCountZeroTests returns null values where numeric values are expected, which breaks deserialization:
{
  "took" : 51,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "extended_stats#commit_stats" : {
      "count" : 0,
      "min" : null,
      "max" : null,
      "avg" : null,
      "sum" : 0.0,
      "sum_of_squares" : null,
      "variance" : null,
      "variance_population" : null,
      "variance_sampling" : null,
      "std_deviation" : null,
      "std_deviation_population" : null,
      "std_deviation_sampling" : null,
      "std_deviation_bounds" : {
        "upper" : null,
        "lower" : null
      }
    }
  }
}

Investigate if this is a change in how extended stats are emitted

Looks like this is a result of two new fields added
      "variance_population" : null,
      "variance_sampling" : null,

Related to https://github.com/elastic/elasticsearch/pull/49782


[.Net] Add Goolge gemini

Why are these changes needed?

Related issue number
#2851 task 1,2,3,4,5

Checks

[ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
[ ] I've added tests (if relevant) corresponding to the changes introduced in this PR.
[ ] I've made sure all auto checks have passed.

@hsm207 could you please review the examples under sample/AutoGen.Gemini.Sample

@LittleLittleCloud I ran all the Vertex-related samples. Here is what I found:

All samples run as expected except Image_Chat_With_Vertex_Gemini
For Image_Chat_With_Vertex_Gemini, the path to the image is wrong, it should be:

var imagePath = Path.Combine("..", "..", "resource", "images", "background.png");

For Image_Chat_With_Vertex_Gemini, both images in the /resource/images folder give the following error:

Unhandled exception. System.AggregateException: One or more errors occurred. (Status(StatusCode="InvalidArgument", Detail="Provided image is not valid."))
 ---> Grpc.Core.RpcException: Status(StatusCode="InvalidArgument", Detail="Provided image is not valid.")
   at Grpc.Net.Client.Internal.HttpContentClientStreamReader`2.MoveNextCore(CancellationToken cancellationToken)
   at Google.Api.Gax.Grpc.AsyncResponseStream`1.MoveNextAsync(CancellationToken cancellationToken)
   at AutoGen.Gemini.GeminiChatAgent.GenerateStreamingReplyAsync(IEnumerable`1 messages, GenerateReplyOptions options, CancellationToken cancellationToken)+MoveNext() in /workspaces/autogen/dotnet/src/AutoGen.Gemini/GeminiChatAgent.cs:line 151
   at AutoGen.Gemini.GeminiChatAgent.GenerateStreamingReplyAsync(IEnumerable`1 messages, GenerateReplyOptions options, CancellationToken cancellationToken)+MoveNext() in /workspaces/autogen/dotnet/src/AutoGen.Gemini/GeminiChatAgent.cs:line 151
   at AutoGen.Gemini.GeminiChatAgent.GenerateStreamingReplyAsync(IEnumerable`1 messages, GenerateReplyOptions options, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at AutoGen.Gemini.Middleware.GeminiMessageConnector.InvokeAsync(MiddlewareContext context, IStreamingAgent agent, CancellationToken cancellationToken)+MoveNext() in /workspaces/autogen/dotnet/src/AutoGen.Gemini/Middleware/GeminiMessageConnector.cs:line 48
   at AutoGen.Gemini.Middleware.GeminiMessageConnector.InvokeAsync(MiddlewareContext context, IStreamingAgent agent, CancellationToken cancellationToken)+MoveNext() in /workspaces/autogen/dotnet/src/AutoGen.Gemini/Middleware/GeminiMessageConnector.cs:line 48
   at AutoGen.Gemini.Middleware.GeminiMessageConnector.InvokeAsync(MiddlewareContext context, IStreamingAgent agent, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at AutoGen.Core.PrintMessageMiddleware.InvokeAsync(MiddlewareContext context, IStreamingAgent agent, CancellationToken cancellationToken)+MoveNext() in /workspaces/autogen/dotnet/src/AutoGen.Core/Middleware/PrintMessageMiddleware.cs:line 54
   at AutoGen.Core.PrintMessageMiddleware.InvokeAsync(MiddlewareContext context, IStreamingAgent agent, CancellationToken cancellationToken)+MoveNext() in /workspaces/autogen/dotnet/src/AutoGen.Core/Middleware/PrintMessageMiddleware.cs:line 54
   at AutoGen.Core.PrintMessageMiddleware.InvokeAsync(MiddlewareContext context, IStreamingAgent agent, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at AutoGen.Core.PrintMessageMiddleware.InvokeAsync(MiddlewareContext context, IAgent agent, CancellationToken cancellationToken) in /workspaces/autogen/dotnet/src/AutoGen.Core/Middleware/PrintMessageMiddleware.cs:line 24
   at AutoGen.Core.PrintMessageMiddleware.InvokeAsync(MiddlewareContext context, IAgent agent, CancellationToken cancellationToken) in /workspaces/autogen/dotnet/src/AutoGen.Core/Middleware/PrintMessageMiddleware.cs:line 24
   at AutoGen.Core.AgentExtension.SendAsync(IAgent agent, IMessage message, IEnumerable`1 chatHistory, CancellationToken ct) in /workspaces/autogen/dotnet/src/AutoGen.Core/Extension/AgentExtension.cs:line 39
   at AutoGen.Core.AgentExtension.SendAsync(IAgent agent, String message, IEnumerable`1 chatHistory, CancellationToken ct) in /workspaces/autogen/dotnet/src/AutoGen.Core/Extension/AgentExtension.cs:line 59
   at AutoGen.Gemini.Sample.Image_Chat_With_Vertex_Gemini.RunAsync() in /workspaces/autogen/dotnet/sample/AutoGen.Gemini.Sample/Image_Chat_With_Vertex_Gemini.cs:line 37
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at Program.<Main>$(String[] args) in /workspaces/autogen/dotnet/sample/AutoGen.Gemini.Sample/Program.cs:line 9

I tried with a different image and in jpg format, and it works.
Thanks for making this PR so quickly!

@hsm207 Thanks for the feedback, I'll take a look

For Image_Chat_With_Vertex_Gemini, the path to the image is wrong

Looks like you are running this example on a unix system, I wonder this error might be because the resource path is configured in a windows-path-like way (forward \) in Directory.Build.Props, I'll take a look

@LittleLittleCloud I was running it inside the full devcontainer.

I was running it inside the full devcontainer. I think I got that error because I ran the examples from the AutoGen.Gemini.Sample folder and am guessing you did it from the dotnet folder?

Oh you are right, that's probably right. Because if you run the project under sample folder, then the Directory.Build.Props won't taken into effect

I tried with a different image and in jpg format, and it works.

It's probably because of git lfs so the image in repo is actually a hash code instead of an image?

It's probably because of git lfs so the image in repo is actually a hash code instead of an image?

 I did not notice it was just a hash code before. After pulling all the lfs objects, the sample with those image works.


IMB-68: Add CVE exceptions file

What?
ALLOW_CVE_LIST_FILE set to the path to a file containing a list of allowed CVEs
Place a text file that contains a list of vulnerabilities (CVEs) and comments elsewhere within your repository however you must set environment variable ALLOW_CVE_LIST_FILE to the path of the text file relative to the root of your repository.
Why?
Path to file containing a list of allowed CVEs. Listing CVEs in a file with ALLOW_CVE_LIST_FILE. Trivy will scan images, file systems and repositories for any vulnerabilities and issues. Trivy is supported by ACP team
How?
Add missing CVE Exceptions file for trivy.
The Common Vulnerabilities and Exposures (CVE) system establishes a standard for reporting and tracking vulnerabilities. CVE Identifiers are assigned to vulnerabilities to make it easier to share and track information about these issues.
Testing?

I agree with you @jamiecarterHO . I followed html-pdf-converter where this file is in the open repo.
We definitely need to store in secure/private repo like hof-services-config. I will add a folder in hof-services-config and add the CVE exception file there.


Add empty constructor for DubboLifecycleComponentApplicationListener

What is the purpose of the change
add empty constructor for DubboLifecycleComponentApplicationListener
related issue #7347

DubboBootstrapApplicationListener also need an empty constructor for compatible with old version

done


Added Explore Page

Added Explore page but messed up pages.

Fixed home page formatting and added explore page formatting


Fix: Server Init

when the server restarts, it creates a gateway. The workspace Id is not matching in the gateway.

We moved to a different architecture


Default compiler standard

Also, with regards to the temporary file: we could distribute the "prepared" C/C++ code as a resource with the CodeChecker install, instead of creating them at runtime.

I tried to keep these C++ codes in separate files but it was not possible to reach them from the test code.

@bruntib That is an ongoing issue (that the tests can rarely pull configuration and other stuff from the deployed environment), see #787.
Until then, I think still putting these into separate files (e.g. config or vendor?) and making the make test command unconditionally copy them over to some well-defined location is a good workaround. When we have some time, as part of #787, we will figure something better out.

Was just an idea, this workaround is good enough for me. However, I think it would be better if the code, if it stays in the Python file, gets moved out of the function's scope? It is... essentially a static const of this function, it is a resource, it is also very long code, it should reside amongst some top-level declarations.

The outer scope is global scope which we try to fight against, even in this ticket for another variable. Would you suggest to place it there? By the way in an other language I would chose static const as it supports the rule of thumb of keeping the variables in the innermost possible scope. We have even a checker for this :)


Bittrex Trading Error

2017-11-18 06:24:00  0.168370 BCC-BTC   -1.33%      728   58 ++      0.0991   0.0573   +1.98%  0.02883648 BCC  0.00000994 BTC   -9.56%   -8.43%
2017-11-18 06:26:00  0.167617 BCC-BTC   -0.45%      656   55 ++      0.0691   0.0483   +1.53%  0.02883648 BCC  0.00000994 BTC   -9.97%   -8.43%
2017-11-18 06:28:00  0.168500 BCC-BTC   +0.53%      283   58 ++      0.0941   0.0504   +2.06%  0.02883648 BCC  0.00000994 BTC   -9.49%   -8.43%
server_1        | undefined
server_1        | /app/node_modules/numbro/numbro.js:1111
server_1        |         var parts = x.toString().split('.');
server_1        |                       ^
server_1        |
server_1        | TypeError: Cannot read property 'toString' of undefined
server_1        |     at multiplier (/app/node_modules/numbro/numbro.js:1111:23)
server_1        |     at /app/node_modules/numbro/numbro.js:1127:22
server_1        |     at Array.reduce ()
server_1        |     at correctionFactor (/app/node_modules/numbro/numbro.js:1125:21)
server_1        |     at Numbro.add (/app/node_modules/numbro/numbro.js:1197:47)
server_1        |     at /app/commands/trade.js:391:78
server_1        |     at /app/lib/engine.js:192:14
server_1        |     at /app/extensions/exchanges/bittrex/exchange.js:113:18
server_1        |     at Request._callback (/app/node_modules/node.bittrex.api/node.bittrex.api.js:112:25)
server_1        |     at Request.self.callback (/app/node_modules/request/request.js:186:22)
server_1        |     at emitTwo (events.js:135:13)
server_1        |     at Request.emit (events.js:224:7)
server_1        |     at Request. (/app/node_modules/request/request.js:1060:10)
server_1        |     at emitOne (events.js:125:13)
server_1        |     at Request.emit (events.js:221:7)
server_1        |     at IncomingMessage. (/app/node_modules/request/request.js:980:12)
server_1        |     at Object.onceWrapper (events.js:322:30)
server_1        |     at emitNone (events.js:120:20)
server_1        |     at IncomingMessage.emit (events.js:218:7)
server_1        |     at endReadableNT (_stream_readable.js:1054:12)
server_1        |     at _combinedTickCallback (internal/process/next_tick.js:138:11)
server_1        |     at process._tickCallback (internal/process/next_tick.js:180:9)

There is a previous bug listed #435 that outlines the problem.  For whatever reason, if you point to an alternate config file, the API keys are always read from conf.js.  I haven't looked through the code yet, but a simple solution is to just name your config "conf.js".  I've tested it myself with Bittrex and it works as expected.

Are you still experiencing this problem.  If not, consider closing this.

I have the same problem with binance.
A few hours ago the bot has worked.
I have installed the latest version of zenbot
c:\zenbot>node zenbot.js sim --strategy cci_srsi binance.ETC-BTC
If you have questions about this strategy, contact me... @talvasconcelos
(node:9316) DeprecationWarning: collection.ensureIndex is deprecated. Use createIndexes instead.
2018-08-06 04:39:00  0.00245900 ETC-BTC   +0.20%     8864     - 46.25      -213   -->         buyc:\zenbot\node_modules\lolex\src\lolex-src.js:663
throw firstException;
^
TypeError: Cannot read property 'toString' of null
at multiplier (c:\zenbot\node_modules\numbro\numbro.js:1123:23)
at c:\zenbot\node_modules\numbro\numbro.js:1139:22
at Array.reduce ()
at correctionFactor (c:\zenbot\node_modules\numbro\numbro.js:1137:21)
at Numbro.add (c:\zenbot\node_modules\numbro\numbro.js:1218:47)
at c:\zenbot\lib\engine.js:362:61
at c:\zenbot\lib\engine.js:235:9
at c:\zenbot\lib\engine.js:304:7
at c:\zenbot\extensions\exchanges\sim\exchange.js:66:18
at callTimer (c:\zenbot\node_modules\lolex\src\lolex-src.js:356:24)
at Object.tick (c:\zenbot\node_modules\lolex\src\lolex-src.js:622:25)
at withOnPeriod (c:\zenbot\lib\engine.js:767:19)
at onTrade (c:\zenbot\lib\engine.js:934:7)
at c:\zenbot\lib\engine.js:895:5
at c:\zenbot\node_modules\async\dist\async.js:4096:9
at Object.process (c:\zenbot\node_modules\async\dist\async.js:2344:17)
at c:\zenbot\node_modules\async\dist\async.js:2252:19
at Immediate._onImmediate (c:\zenbot\node_modules\async\dist\async.js:119:16)
at runCallback (timers.js:810:20)
at tryOnImmediate (timers.js:768:5)
at processImmediate [as _immediateCallback] (timers.js:745:5)
c:\zenbot>zenbot -v
c:\zenbot>node zenbot.js -v
error: unknown option `-v'
c:\zenbot>npm --v
5.6.0
c:\zenbot>


[DNM] Reproduce some bug

Reproduce in velox-backend-ubuntu2004-test.
Correct result: {7, 7, 7}
Velox result: {7, 7, 7}
After convert RowVector to GlutenBatch: {null, 7, 7} or core

@jinchengchenghh @rui-mo @zhztheplayer @zhouyuan

@Yohahaha Thank you. I will take a look later.

This is caused by the last operator C2R, and toArray uses the memory freed by the iterator. @JkSelf has a fix maybe can resolve the problem.

@Yohahaha   This is a bug for spark 3.2 and 3.3. And we have merged PR#40958 into spark 3.3 to fix this issue. I will port this fix in Gluten later. Thanks.

Good to know this!


Add Link header for the ArchivalGroup of a resource

JIRA Ticket: https://fedora-repository.atlassian.net/browse/FCREPO-3295
What does this Pull Request do?
Searches the ancestry of a resource to see if any parent is an Archival Group and adds a Link header if one exists
How should this be tested?

Create an Archival Group
Create some resources underneath the Archival Group
Perform some HEAD requests on the child resources and check for the Archival Group Link header

Interested parties
@fcrepo/committers

I don't doubt that this works, however, I fear that it will be quite slow. Archival part resource actually contain a reference to their containing archival group resource in their ResourceHeaders. It should be much more efficient to use this field to solve this issue. I would expect that exposing it on FedoraResource would be fine.

I don't doubt that this works, however, I fear that it will be quite slow. Archival part resource actually contain a reference to their containing archival group resource in their ResourceHeaders. It should be much more efficient to use this field to solve this issue. I would expect that exposing it on FedoraResource would be fine.

It looks like isArchivalGroup already exists as a field in FedoraResourceImpl as well, so this makes sense. I'll have an update ready soon with changes reflecting this.
I'm curious though, was the main concern around resource.getTypes()? I had assumed these would have been populated when the FedoraResource was created but looking at the impl this doesn't appear to be the case.

I'm curious though, was the main concern around resource.getTypes()? I had assumed these would have been populated when the FedoraResource was created but looking at the impl this doesn't appear to be the case.

No, I was referencing getArchivalGroupId(). So, you ask the resource what archival group it is part of rather than traversing up the resource's ancestors.

No, I was referencing getArchivalGroupId(). So, you ask the resource what archival group it is part of rather than traversing up the resource's ancestors.

I completely missed that from the ResourceHeaders, got a bit too excited when I saw the isArchiveGroup boolean already in the FedoraResourceImpl. Updated again to expose the archivalGroupId if it exists.


DbManager.SetIdentityFields(false) doesn't work for Guids

Can't use DBManager.SetIdentityFields(false) to seed database tables with specific GUID primary keys.  The rows appear inserted with database generated IDs.
I've replicated the issue in a test harness found in this repo:
https://github.com/YauGoh/EffortGuid.git

Hello @YauGoh ,
Thank a lot for your project, I will try to make my developer look at it tomorrow.
Best Regards,
Jonathan

Performance Libraries
context.BulkInsert(list, options => options.BatchSize = 1000);
Entity Framework Extensions • Entity Framework Classic • Bulk Operations •  Dapper Plus •  LinqToSql Plus
Runtime Evaluation
Eval.Execute("x + y", new {x = 1, y = 2}); // return 3
C# Eval Function • SQL Eval Function

Hi, Are there any updates on this issue?

Hello @YauGoh ,
Thank for the reminder, it looks the pull has been done more than one month ago: https://github.com/zzzprojects/EntityFramework-Effort/pull/145 but I forgot to review and merge it.
I will talk with my developer this week to try to complete it.
Best Regards,
Jonathan

Hello @YauGoh ,
Sorry again for the delay,
The pull has been merged.
Could you try the latest version v2.0.4 and let me know if everything works as expected?
Best Regards,
Jonathan


Renamed adapter file to match django convention.

Renamed the adapter.py file to adapters.py to match the django naming convention.
It also makes sense since there are two different adapters in that file (normal and social).

:+1:
Could you take a look at #488 again? Once you have that updated I can merge both PRs. That makes it easier when editing the history.
Thank you!

Sure thing..
✓ Added account registration switch to environment example file.


Fix getPixelFromCoordinate function (close #23)

Hi @aarranz,
It would be great if you could review this PR.
Thank you.

Coverage increased (+0.005%) to 99.216% when pulling 20472949f5b57c7f47696a99dc64ef7ab7302666 on fisuda:fix/getPixelFromCoordinate into bf3d5c93292f5e23a20396cc39373aaca02cadfe on Wirecloud:develop.

Thank you very much! Merged :smiley:


Form Field and Filter to parse ISO-8601 timestamps

PR to Fix #189.
Todo:

[x] Form field and Filter
[x] Sufficient test cases
[x] Docs

@carltongibson feel free to give me a nudge in a few days if I've not helped out with review etc. :)

just tried this out and the new IsoDateTimeFilter works for me. Nice job!

@mcastle Thanks for the feedback. Everything should work but, well, timezones... :-)
I will add a few more test cases for this and write up the docs, and sort Django 1.4 — grrrrr... — this week. Time for a new release I think.

Neat :smile: - will we be able to map to this by default from REST framework's filter class then?

@tomchristie — Yeah, why not. I'm just adding a test or two now and some docs (then I was going to ask you to review)
I'll look at DRF and open a matching PR for that change (and if we time it right update to the new django-filter version)

Rockin'
Of course we'll need to make sure it falls back to the standard field if this one doesn't exist.

Having DRF use the new field would be a good integration test as well

Very true.

OK, @tomchristie — can you just cast your eye on this one. It's got some tests, some docs, and I'm inclined to stick. (Ta!)
I'll open a new ticket to introduce a DRF-friendly base FilterSet class: people want this, but they also want more JavaScript friendly query parameter parsing — and no doubt other things too :-) — so it'll be worth the extra indirection.

@carltongibson Can you give a complete example how to use IsoDateTimeField? I read the docs and couldn't understand. In my case I've got a django model with DateTimeField and a filter for it that extends django_filters.FilterSet.

@leonprou See the example in the docs: https://github.com/carltongibson/django-filter/blob/develop/docs/ref/filters.txt#L241-L247
If that's not enough you're going to have to explain what you're not able to get working.

@carltongibson thank you for your response!

@carltongibson thank you for the quick response!
the example did help, but now I struggle to use the gte tag, my filter looks like this:
class MyClass(django_filters.FilterSet):
    last_update_date = IsoDateTimeFilter()
    class Meta:
        model = MyClass
        fields = {
            'last_update_date': ['exact', 'gte']
        }

it works for exact date but not for gte

You can't use the metaclass and the explicit filter declaration. You need to explicitly define two filters, one for each lookup. Again, this is in the docs: https://django-filter.readthedocs.io/en/latest/usage.html#declaring-filters

Thanks, it's working!


Doc buffer single side

Added doc in the manual about buffer function single_sided option merged in #806 with doc at API level but not in the manual

Coverage remained the same at 85.133% when pulling 1c07296aa1d235eec7ba81bef8a53dd2e97b821e on ThomasG77:doc-buffer-single-side into 8908a36b910b34485bb4ba2fbd9856d73bbeb84f on Toblerity:master.

Before

After

Thank you @ThomasG77 ! Eventually we may want to build docs for commits, but that isn't on the roadmap yet.


Update configuration and workflow for Release Drafter

Addresses WordPress/openverse#20.
This PR updates the existing configuration for release drafter.
Changes:

Added humanized names for worflow job and step
Updated labels used to categorize PRs into appropriate sections

I made a teeny tiny fix to something that's been bothering me for months
https://github.com/WordPress/openverse-frontend/pull/71/files#diff-101bec72d0f1f84e7290d113531bbd8c6aa355cdc9f456df255544bc0a2da0ea
Looks good! Reminds me though, we need to bring back issue templates too.


[SPIKE] Testing PR #6322 (build performance update)

This ticket encompasses the work necessary to review PR #6322 (related to #6073) and determine the following:

Does this actually improve performance?  The contributor claims some level of performance improvement, but we need proper benchmarks and the core team should be responsible for building said benchmarks-- especially since initial testing on smaller projects and synthetic benchmarking projects doesn't seem to show much of an improvement.
Is this algorithm the right choice to use across the board?  Does it have performance characteristics that fit everyone's needs?

Once this spike determines this work is suitable for what we need here we can work with the contributor to correct some code issues and add some testing.  If it doesn't we should probably prioritize putting dbt engineering hours into this since it's obviously been an issue for some time and judging by slack conversations, there's a real paint point to be solved here.
This ticket ~may~ should also drive us to get better performance monitoring setup around the build command.
Slack refs:

https://dbt-labs.slack.com/archives/C04191CFLK0/p1673281639142599?thread_ts=1672911813.317039&cid=C04191CFLK0
https://dbt-labs.slack.com/archives/C0131TY7EEA/p1673291456196899

Prior art for performance benchmarking + regression testing: https://github.com/dbt-labs/dbt-core/tree/main/performance
With partial parsing enabled, I believe we could use this top-level command to reliably isolate the time associated with compiling a DAG with "extra test edges": dbt build --exclude fqn:*

Scope of this ticket is just to decide if want to merge this PR or not. Additional foundational performance improvements are out of scope for now.

Using dbt's built-in timing info (thanks @dbeatty10!), I ran dbt --no-partial-parse compile on a gnarly DAG sized 8468 models, 17103 tests. It took 14m4s to run on dbt-core#tag/v1.3.2 code, and 16m3s to run on dbt-core#tag/v1.3.2 with #6322 patch applied to dbt/graph/graph.py.
Looking into the charts below, we see that indeed the changed code in #6322 seems to have added time.


How does % max hp work?

Like if i have a few perk with exactly number like +10 +3,do perk increase % max hp work on those bonus health or only increase base health?

This is TW math so 3% mean flat +3 :)

It's relative to the base health of '100' for some reason, so like @peppermor said, 3% is 3.

It's relative to the base health of '100' for some reason, so like @peppermor said, 3% is 3.
Really sad,if i have 100 hp with 100 bonus hp i should have 206 not 203 :(

I think in the future base health might vary. I dunno. Seems like it should stack after adds, but not with the methods they wrote. We're trying to stick with TW's intentions.
Maybe in the future base health will be affected by starvation / age / illness / difficulty?

I think in the future base health might vary. I dunno. Seems like it should stack after adds, but not with the methods they wrote. We're trying to stick with TW's intentions.
Maybe in the future base health will be affected by starvation / age / illness / difficulty?

That would be fun haha


[AMBARI-23779] - Limitation for number of URL paramater value length

What changes were proposed in this pull request?
The goal of these changes is to turn the GET request to POST where the parameters possibly can reach the character limit what the browsers can handle.
How was this patch tested?
Manually and with unit tests (267/267 success)
Please review Ambari Contributing Guide before opening a pull request.
@aBabiichuk please review it. Thanks.

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/2194/
Test PASSed.


Klipper svclb not working in new version of K3s that we are using

Trying to deploy Big Bang using Zarf v0.19.5, and the klipper svclb service isn't working. Still investigating. What I'm seeing now is that the service public-ingressgateway with type LoadBalancer is stuck in <pending> for receiving an external IP. When I roll back to K3s v1.21.6+k3s1 it works fine, though I do have to add docker.io/rancher/klipper-lb:v0.2.0 as a specified image since the Zarf mutating webhook modifies the pod spec to point at the internal docker registry.

Could you dump the deployment (or equivalent) yaml that is in the running cluster?

not sure what you mean

kubectl -n  get   -o yaml
I didn't see what type of resource this podspec is in (deployment, statefulset, daemonset, etc)

K. I'm about to deploy the working version (using the old version of k3s), will grab that. Will then deploy the non-working version in a bit to grab that.

Non-working example, using Zarf v0.19.5 with the built in K3s v1.24.1+k3s1.
Of note is that the daemonset for reasons unknown says the "desired" number of replicas is 0.

Daemonset Yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  annotations:
    deprecated.daemonset.template.generation: "1"
    objectset.rio.cattle.io/applied: H4sIAAAAAAAA/8SWTW/jNhCG/0oxZ0mR/JHYAnoIkhyCdhPD9vayMIIRNbJZU6RAjrQxAv33grKTOK0/gmK7PRkkh6847zPm8AWwkn+QddJoSAGryl00CQSwljqHFG6RSqNnxBBASYw5MkL6Aqi1YWRptPNDk/1Jgh1xZKWJBDIriqS5kF7DNUIYzdYoRRaCo8HmuyYbLps1pHDRJMEvv0md/zoj20hBZ/dpLAlSqOpMSRFKvbTk3BKZvuPmU5tdhcIrSMfShG7jmEpoAxCWukTnsiTHWFaQ6lqpABRmpE6mv0K3ghR6o+SyiOkKY8opGxfDsehno1gUl71YjK/6g+FlgZdDb/sHr6J13+2paZOTI0WCjYUUClSO/AF3mbtGqCw8lv/RDAPoLJhSQZa0IAfpt79VRVcQewhTtjW9lcg7oDMI6q4akl48HvbEVTjIx+NwECe9MKM+hnlvlORFPBrlSQHtog3AVSS8u+85v0CJLFa/vxmPVXUm87YNgKmsFDJ1AntF/Amy5z9wmplrxElX2r08sWZTmlrzztFrIfxobtakIe1wbzGg1GS3nEg33e/uI7PpzdPkcTqHABpUtZ9KhnEvgTZ4i7m9m82fJtPH+eNe1Pxm8s+YzwjdT2b7MXE06Ee9fi8aD6FdBCBLXPoFi1qsyF6slawqsqHK0iaO+tEQdjGTWqmJUVJsIIX74sHwxJIjze91pbKQRRVuzxFAZSxvTXjzZGIsQ9oFBLAyjj9MHJGxho0w6tWERQCWnKlt91d48XxI1Fby5sZopmfuKgcrzKSSLGlbJnkO6Td4uJs/Xd9+uX+ARdt6p87jGcU/gs0BlZ8PZhSfojKK95H40aHd/y+MwaD/I2gckvn5OPwpTvAYDPr7QLrhQYH/DMkiADaK7Oszwl9nRUGCIYUHMxMrymvl+8qafLK+/YXWKIrWdUZWE5PzV2yJjrfvisprdb3x7lk6dh2EfyO5u87DSqGmo8pbjRsrWQpU13lutHvUanN4w8Jf9XWVI9OMLTItN94Z3zWkXn7tFrYd7vmrxgalwkwRpIlvYJvKc5l+iO1aByPXncOitpY0P9RlRvY10RzSOICcnLSUH1rS3dwX6dyB6SlhvoE0btu/AgAA///xvZR2JgoAAA
    objectset.rio.cattle.io/id: svccontroller
    objectset.rio.cattle.io/owner-gvk: /v1, Kind=Service
    objectset.rio.cattle.io/owner-name: public-ingressgateway
    objectset.rio.cattle.io/owner-namespace: istio-system
  creationTimestamp: "2022-06-24T15:20:29Z"
  generation: 1
  labels:
    objectset.rio.cattle.io/hash: 2816f0e7a0edeb9f59c3b80cf620c973456fa651
    svccontroller.k3s.cattle.io/nodeselector: "false"
  name: svclb-public-ingressgateway
  namespace: istio-system
  ownerReferences:
  - apiVersion: v1
    controller: true
    kind: Service
    name: public-ingressgateway
    uid: 120952c7-4d99-4012-be3a-d281df088d1f
  resourceVersion: "4480"
  uid: b26ac30f-8173-42d0-9836-1bd56fd09550
spec:
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: svclb-public-ingressgateway
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: svclb-public-ingressgateway
        svccontroller.k3s.cattle.io/svcname: public-ingressgateway
    spec:
      automountServiceAccountToken: false
      containers:
      - env:
        - name: SRC_PORT
          value: "15021"
        - name: DEST_PROTO
          value: TCP
        - name: DEST_PORT
          value: "15021"
        - name: DEST_IPS
          value: <IP_ADDRESS>
        image: rancher/klipper-lb:v0.3.5
        imagePullPolicy: IfNotPresent
        name: lb-tcp-15021
        ports:
        - containerPort: 15021
          hostPort: 15021
          name: lb-tcp-15021
          protocol: TCP
        resources: {}
        securityContext:
          capabilities:
            add:
            - NET_ADMIN
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      - env:
        - name: SRC_PORT
          value: "80"
        - name: DEST_PROTO
          value: TCP
        - name: DEST_PORT
          value: "80"
        - name: DEST_IPS
          value: <IP_ADDRESS>
        image: rancher/klipper-lb:v0.3.5
        imagePullPolicy: IfNotPresent
        name: lb-tcp-80
        ports:
        - containerPort: 80
          hostPort: 80
          name: lb-tcp-80
          protocol: TCP
        resources: {}
        securityContext:
          capabilities:
            add:
            - NET_ADMIN
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      - env:
        - name: SRC_PORT
          value: "443"
        - name: DEST_PROTO
          value: TCP
        - name: DEST_PORT
          value: "443"
        - name: DEST_IPS
          value: <IP_ADDRESS>
        image: rancher/klipper-lb:v0.3.5
        imagePullPolicy: IfNotPresent
        name: lb-tcp-443
        ports:
        - containerPort: 443
          hostPort: 443
          name: lb-tcp-443
          protocol: TCP
        resources: {}
        securityContext:
          capabilities:
            add:
            - NET_ADMIN
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
      tolerations:
      - effect: NoSchedule
        key: node-role.kubernetes.io/master
        operator: Exists
      - effect: NoSchedule
        key: node-role.kubernetes.io/control-plane
        operator: Exists
      - key: CriticalAddonsOnly
        operator: Exists
  updateStrategy:
    rollingUpdate:
      maxSurge: 0
      maxUnavailable: 1
    type: RollingUpdate
status:
  currentNumberScheduled: 0
  desiredNumberScheduled: 0
  numberMisscheduled: 0
  numberReady: 0

No pod yaml, it never made one.

The agent ignores existing namespaces and can also ignores individual resources if they have the correct label. The issue here is we want to mutate the istio-system namespace for some things but not other things.  I see 2 options (don't know if either are possible off the top of my head): have the daemonset deploy on a different namespace (could be kube-system), or add the correct label to the daemonsets to tell the zarf agent to ignore it.

That makes sense. I wonder though why in the older version of K3s the daemonset's pod is created properly (just in a failed state unless the Zarf registry contains the image it needs) while the new version of K3s never creates the pod in the first place

Yeah I'm not sure, super odd.

I wonder if this could be the reason
https://twitter.com/karlkfi/status/1525316811364765696?s=21&t=ayBPQRH44aTzcmm30Levsw

Sounds like there is a flag that can be set to get the old behavior back
https://twitter.com/karlkfi/status/1525366754213474305?s=21&t=RmdJUK4dmDA9AKIRaJO7lQ

@jeff-mccoy can we add this to the GA milestone?

Update: Looks like the issue is likely to lie with Gatekeeper policies when deploying Big Bang rather than an issue with the version of K3s. In K3s v1.24 it wants to deploy daemonset pods using NodePort and Gatekeeper blocks it given the current configuration in the Big Bang package in this repo


StreamingCommand failed when input contains non-ascii character

Environment:

splunk8.0
splunk sdk 1.6.11
python 3

When I create StreamingCommand   "testcommand" with following code:
#!/usr/bin/env python
#-*- coding: utf-8 -*-
import sys
from splunklib.searchcommands import \
    dispatch, StreamingCommand, Configuration, Option, validators
import splunk
@Configuration()
class TestCommand(StreamingCommand):
    def stream(self, events):
        for event in events:
            yield event
dispatch(TestCommand, sys.argv, sys.stdin, sys.stdout, __name__)

testcommand will hang when I use following SPL:
sourcetype=XXX| search url = "http://例子.卷筒纸" | testcommand
There has follwing error log in splunkd.log:

10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: During handling of the above exception, another exception occurred:
10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: Traceback (most recent call last):
10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: File "C:\Program Files\Splunk\etc\apps\XXXX\bin\testcommand.py", line 22, in
10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: dispatch(TestCommand, sys.argv, sys.stdin, sys.stdout, name)
10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: File "C:\Program Files\Splunk\etc\apps\XXXX\bin\splunklib\searchcommands\search_command.py", line 1118, in dispatch
10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: command_class().process(argv, input_file, output_file)
10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: File "C:\Program Files\Splunk\etc\apps\XXXX\bin\splunklib\searchcommands\search_command.py", line 435, in process
10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: self._process_protocol_v2(argv, ifile, ofile)
10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: File "C:\Program Files\Splunk\etc\apps\XXXX\bin\splunklib\searchcommands\search_command.py", line 787, in _process_protocol_v2
10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: self.finish()
10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: File "C:\Program Files\Splunk\etc\apps\XXXX\bin\splunklib\searchcommands\search_command.py", line 393, in finish
10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: self._record_writer.flush(finished=True)
10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: File "C:\Program Files\Splunk\etc\apps\XXXX\bin\splunklib\searchcommands\internals.py", line 775, in flush
10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: self._write_chunk(metadata, self._buffer.getvalue())
10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: File "C:\Program Files\Splunk\etc\apps\XXXX\bin\splunklib\searchcommands\internals.py", line 820, in _write_chunk
10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: self._ofile.flush()
10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: OSError: [Errno 22] Invalid argument
10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='utf-8'>
10-25-2019 16:34:42.808 +0800 ERROR ChunkedExternProcessor - stderr: OSError: [Errno 22] Invalid argument

When previous output contains non-ascii character,
it seems this StreamingCommand blocked in following code "ifile.read(body_length)"
try:
            if body_length > 0:
                body = ifile.read(body_length)
        except Exception as error:
            raise RuntimeError('Failed to read body of length {}: {}'.format(body_length, error))

in function "_read_chunk " in class SearchCommand.

When I use search command protocol version 1, this issue is not replicate
[testcommand]
filename=testcommand.py
enableheader = true
outputheader = true
requires_srinfo = true
stderr_dest = message
supports_getinfo = true
supports_rawargs = true
supports_multivalues = true

I actually JUST encountered this issue! It took me forever to narrow down what was happening. I'm not sure if it's a problem with the sdk so much as it is with splunkd.
I have a command I'm doing a POC with, and I tested it with:
| makeresults | eval string="hey guys" | formatasjson string
which comes through stdin as:
chunked 1.0,36,66
{"action":"execute","finished":true}
"_time",string,"__mv__time","__mv_string"\n1590694186,"hey guys",,
But then when I run the same search as with a non-ascii character replacement:
| makeresults | eval string="héy guys" | formatasjson string
I get an +1 content length from the header:
chunked 1.0,36,67
{"action":"execute","finished":true}
"_time",string,"__mv__time","__mv_string"\n1590694057,"héy guys",,
The sdk (quite naturally) hangs at:
splunklib/searchcommands/search_command.py:
888.  try:
889.  ....if body_length > 0:
890.  ........body = ifile.read(body_length)
Because it's waiting for that 67th character that will never come.
Right now I'm just trying to make a band-aid approach to monitor for a "deceptive header." I'm not sure yet if it's one-off for every one non-ascii character, or if it's just one-off period. This is just an uninformed guess, but I'm wondering if it has to do with Splunk using Python 2 (which requires the "u" prefix before a unicode string). That's the only single-character difference I can think of between ascii and unicode at the moment, but I wouldn't know why Splunk would be using a repr operation on a string.

I actually JUST encountered this issue! ...
... I'm just trying to make a band-aid approach ...

This worked for me:
splunklib/searchcommands/search_command.py:
  888. try:
- 889. ....if body_length > 0:
+ 889. ....body_saved = body_length
- 890. ........body = ifile.read(body_length)
+ 890. ........while body_saved > 0:
+ 891. ............char = ifile.read(1)
+ 892. ............body_saved -= 2 if char.encode('ascii', errors='ignore') == b'' else 1
+ 893. ............body += char
Sorry that's not very copy/paste friendly. Basically I just changed the ifile.read(body_length) line (and the preceding line) to:
body_saved = body_length
while body_saved > 0:
    char = ifile.read(1)
    body_saved -= 2 if char.encode('ascii', errors='ignore') == b'' else 1
    body += char

After some testing, I was able to determine that Splunk will consistently add +1 to the body_length value in the header for every non-ascii character. (ascii_char = 1, unicode_char = 2) When I made this fix it became stable and functional.
I can't think of anything that doesn't involve reading it in character by character. I played around with a timeout, but even that needed to read it in one at a time. There's not one-size fits all timeout that ends up being more performant than reading it in that way. (And a read(length) timeout that fails would have to fall back on character by character at that point regardless)
My personal next step is to try and open up a question with Splunk itself on fixing their header bug, but with this I can move on developing my app at least.

Just spent way too long debugging this myself.
The issue is that Splunk platform sends the header in BYTES, but splunklib uses it to read from a TextIOWrapper, which takes CHARS.
So characters that are more than 1 byte (can be 2 or 3 bytes or more) cause the body_length sent to splunklib to be longer in BYTES than in CHARS.  Then when splunklib does .read(), it ought to hit the EOF and return, but it doesn't and just hangs forever.
Sample string that triggers the bug: "Hi 接"
It's 4 characters, but 6 bytes.  Splunk sends a body_length of 6.  Splunklib tries to read 6 CHARACTERS from the stdin (TextIOWrapper), but hangs because there are only 4 characters.  The SPL pipeline hangs and uses its memory and disk space forever.


[WIP] Docker improvements

Features

Two stage dockerbuild using the runnable-created, not cabal-run
due to cabal run, the output was cached until cabal exited, which is now gone
Arguments for GHC and Endemic Version
Some Labels on the Docker File
CI

Problems:

The following error is shown on docker-compose up --build : Endemic_Example_1  | endemic: attempting to use module ‘main:Check.Helpers’ (./Check/Helpers.hs) which is not loaded after which the program exits.

Related Issues: #17 and #20 will be addressed (or discarded) later.

@Tritlo could you have a look at the issue with the Packages?
I also tried to add the check-helpers as a library within endemic.cabal, but that didn't change the error.
I also tried to add the check-helpers as imports / packages in the docker_config.json, but I think that was redundant with what is done in Check.hs

Is the source code of the check-helpers packaged being made available in the docker container?


Refactor data structures

#26 showed that the data is too unstructured to easily work with. With a refactor like moving the ML portion to a seperate project, and communicating over http using kubernetes' dns, while wrapping forecasts and static forecasts in objects that handle different parts of the project is a better solution than to have Database and Forecaster handle everything.
Additionally moving communication within Kubernetes to its own modules would significantly improve the readability of the code
Moving the python part into another repo and containerizing that separately is optional as it goes against the design of the project, but I think it's a better solution
In other words: modularize the hell out of this code

By static forecasts i mean the forecasts set manually in the GUI which isn't implemented yet


compilerOptions.paths in tsconfig isn't respected for TypeScript Astro configs (astro.config.ts)

Astro Info
(Note, this command only worked once I temporarily removed changes to `astro.config.ts`)

Astro                    v4.0.8
Node                     v18.16.0
System                   macOS (arm64)
Package Manager          npm
Output                   static
Adapter                  none
Integrations             none

If this issue only occurs in one browser, which browser is a problem?
No response
Describe the Bug
According to the docs:

TypeScript config file loading is handled using tsm and will respect your project tsconfig options.

However, the following setup doesn't actually compile:
tsconfig.json
{
  "extends": "astro/tsconfigs/strict",
  "compilerOptions": {
    "baseUrl": ".",
    "paths": {
      "@/util/*": ["./src/util/*"],
    }
  },
}

astro.config.ts
import { defineConfig } from 'astro/config';
import { helloWorld } from '@/util/hello-world';
helloWorld();
export default defineConfig({});

src/util/hello-world.ts
export function helloWorld() { console.log('it worked!'); }

The error you get varies by environment but it's essentially something similar to:

Cannot find module '@/util/hello-world' imported from '/absolute/path/to/astro.config.ts'

As soon as you stop using the alias in astro.config.ts, it works as expected:
 import { defineConfig } from 'astro/config';
-import { helloWorld } from '@/util/hello-world';
+import { helloWorld } from './src/util/hello-world';
 helloWorld();
 export default defineConfig({});

This is obviously a contrived example, but I have a more real world case that this affects more dramatically (wiring up a Socket.io server as a custom vite plugin that uses TS code shared between client / server / vite-plugin).
What's the expected result?
The aliases defined in tsconfig.json should work per the docs.
Link to Minimal Reproducible Example
https://stackblitz.com/edit/github-u2tsfo?file=astro.config.ts
Participation

[ ] I am willing to submit a pull request for this issue.

Someone please make a PR and add the necessary sentence so we can start the new year with zero issues!!  😉 🍾

I'm on it

Someone please make a PR and add the necessary sentence so we can start the new year with zero issues!!  😉 🍾

If no one gets to it before I get home and finish my Master's application, I can do it!

@sarah11918 I think we can start the year with zero issues ;)


Make the logo and favicon configurable

as titled, we should also brand them as Fastify

I use this trick to change (prepend) all the CSS so you can use a custom logo, hide the top bar, etc.
h2 + a, section .models, .download-url-wrapper {
  display: none!important;
}

.opblock-get {
  background: #ebeef1!important;
}

.opblock-summary-path, .swagger-ui .info .title, h3 {
  color: #002f6c!important;
}

.topbar {
  background-color: #002f6c!important;
}

.opblock-get .opblock-summary-method {
  background-color: #0048a6!important;
}

.topbar-wrapper {
  height: 45px;
  background-repeat: no-repeat;
  background-size: contain;
  background-image: url("hello.svg");
}

.topbar-wrapper img {
  display: none;
}

function streamToString (stream) {
  const chunks = [];

  return new Promise((resolve, reject) => {
    stream.on('data', (chunk) => chunks.push(Buffer.from(chunk)));
    stream.on('error', (err) => reject(err));
    stream.on('end', () => resolve(Buffer.concat(chunks).toString('utf8')));
  });
}

fastify.addHook('onSend', async (req, res, payload) => {
  if (req.url === '/docs/static/swagger-ui.css') {
    const styleBuffer = await fs.readFile(path.join(path.resolve('path', 'to', 'styles.css')));

    return styleBuffer.toString() + (await streamToString(payload));
  }
});

ugh... so you basically ALWAYS patch the swagger-ui.css on the fly :/
It would be better to add a option for customCSS and customJS. Both should be then exposed like static files, and then patch once the index.html from swagger.ui with a style-tag and script tag for the css and js file and than it should be working.
Also we can add an option for favicon and if we set a favicon use https://www.npmjs.com/package/fastify-favicon and if not use favicon.ico like before.

Sure, but right now it's the only way I know. It can be improved, but it will still remain weird 😳


[BUG] Frequent cases with Starting lock renewal messages in log and messages are not delivered

Describe the bug
We have a problem with a Kubernetes service when it consumes messages from a ServiceBus topic.  In some instances when extra processing is performed,  it will stop processing further messages and the log message shows repeated instances of lock renewals.   We are using the latest stable version of Azure Service Bus client for Java - Version 7.15.2.  https://learn.microsoft.com/en-us/java/api/overview/azure/messaging-servicebus-readme?view=azure-java-stable.
Example logs:
2024-04-22 08:56:14,796 INFO [parallel-1] LoggingEventBuilder: {"az.sdk.message":"Starting lock renewal.","isSession":false,"lockToken":"a12845bf-58e9-4650-9d3e-5dba5b234e4f"}
2024-04-22 08:56:14,804 INFO  [reactor-executor-1] LoggingEventBuilder: {"az.sdk.message":"Starting lock renewal.","isSession":false,"lockToken":"a12845bf-58e9-4650-9d3e-5dba5b234e4f","nextExpiration":"2024-04-22T08:56:44.788Z","next":"PT29.983430032S"}
2024-04-22 08:56:34,788 INFO [parallel-1] LoggingEventBuilder: {"az.sdk.message":"Starting lock renewal.","isSession":false,"lockToken":"a12845bf-58e9-4650-9d3e-5dba5b234e4f"}
2024-04-22 08:56:34,794 INFO [reactor-executor-1] LoggingEventBuilder: {"az.sdk.message":"Starting lock renewal.","isSession":false,"lockToken":"a12845bf-58e9-4650-9d3e-5dba5b234e4f","nextExpiration":"2024-04-22T08:57:04.788Z","next":"PT29.993437838S"}

This will often happen in less than 60 seconds from the message consumption even the message is received.  Furthermore the message will not get cleared from the topic in the Azure ServiceBus console.   It requires doing a restart of the Kubernetes service.
The Message Lock Duration is set to 30 seconds in the topic.  If this is changed to 60 seconds then the logs above are spaced approximately 50 seconds apart.
Exception or Stack Trace
We don't observe any exceptions.
To Reproduce
Steps to reproduce the behavior:
I haven't created an exact cut-down / minimal example, however, the code used is trimmed below to describe what we're doing.
Code Snippet
The example code here is split into 4 files.  The first 2 files exist in a common library while the second 2 files use this library in the Kubernetes service.

ServiceBusConsumerHandler - We use ServiceBusProcessorClient and PEEK_LOCK receive mode.

@Builder
public class ServiceBusConsumerFactory<T> {
    private @NonNull String connectionString;
    private @NonNull String topicName;
    private @NonNull String subscriptionName;
    private @NonNull Long maxAutoLockRenewDurationMinutes;
    private @NonNull Class<T> cls;
    private @NonNull Function<Flux<T>, Mono<Void>> processorFunction;

    public ServiceBusConsumer create() {
        final var consumerHandler = ServiceBusConsumerHandler.<T>builder()
                .processor(processorFunction)
                .build();

        ServiceBusProcessorClient serviceBusProcessorClient = new ServiceBusClientBuilder()
                .connectionString(connectionString)
                .processor()
                .topicName(topicName)
                .subscriptionName(subscriptionName)
                .maxAutoLockRenewDuration(Duration.ofMinutes(maxAutoLockRenewDurationMinutes))
                .receiveMode(ServiceBusReceiveMode.PEEK_LOCK)
                .processMessage(consumerHandler::processMessage)
                .processError(consumerHandler::processError)
                .buildProcessorClient();

        return new ServiceBusConsumer(topicName, subscriptionName, serviceBusProcessorClient);
    }
}

ServiceBusConsumerHander.java -  I haven't included the deadLetter()  method below.  It simply looks for exceptions of instance RetryableMessageException.  In this case it calls context.abandon() or else it calls context.deadLetter(option).  There's no evidence of any exceptions in the logs.

@Builder
@Slf4j
public class ServiceBusConsumerHandler<T> {

    private @NonNull Function<Flux<T>, Mono<Void>> processor;

    public void processMessage(ServiceBusReceivedMessageContext context) {
        ServiceBusReceivedMessage message = context.getMessage();
        log.info(
                "Processing service bus message Id: {}, Sequence #: {}}, Contents: {}",
                message.getMessageId(),
                message.getSequenceNumber(),
                message.getBody());

        Flux.just(context)
                .transform(processor)
                .doOnComplete(context::complete)
                .doOnError(ex -> deadLetter(context, ex))
                .then()
                .onErrorComplete()
                .block();
    }

   public void processError(ServiceBusErrorContext context) {
        log.error(
                "Error when receiving message from namespace {} Entity {} error {}",
                context.getFullyQualifiedNamespace(),
                context.getEntityPath(),
                context.getException().getMessage());
    }

ServiceBusConfig.

@Slf4j
@Configuration
public class ServiceBusConfig {

    @Bean
    public ServiceBusConsumer transactionServiceBusConsumer(
            @Autowired NotificationProcessor notificationProcessor,
            @Value("${servicebus.endpoint}") String endpoint,
            @Value("${servicebus.subscription-name}") String subscriptionName,
            @Value("${servicebus.topic-name}") String topicName,
            @Value("${servicebus.max-autolock-renew-duration-minutes}") String maxAutoLockRenewDurationMinutes) {
        final ServiceBusConsumerFactory<EventData> build =
                ServiceBusConsumerFactory.<EventData>builder()
                .connectionString(endpoint)
                .subscriptionName(subscriptionName)
                .topicName(topicName)
                .processorFunction(notificationProcessor)
                .maxAutoLockRenewDurationMinutes(Long.parseLong(maxAutoLockRenewDurationMinutes))
                .cls(EventData.class)
                .build();
        return build.create();
    }
}

maxAutoLockRenewDurationMinutes above is set to 5 minutes.

NotificationProcessor.java

@Slf4j
@Service
public class NotificationProcessor implements Function<Flux<EventData>, Mono<Void>> {

    @Override
    public Mono<Void> apply(Flux<EventData> eventFlux) {
        return eventFlux
                .map(this::toContext) // simple transformation logic
                .flatMap(this::processNotificationEvent)
                .then()
                .doOnError(err -> {
                    log.error("Processing event data failed. ", err);
                });
    }

The code inside processNotificationEvent makes a database query.   In the cases where I can reproduce the problem it also performs a remote procedure call, however, this takes less than 2 seconds.  In these instances, it will quite frequently result in producing the lock messages and the services stops processing the messages.
Expected behavior
I don't expect to see the lock messages occurring and the service should be able to make the remote procedure call.
Screenshots
If applicable, add screenshots to help explain your problem.
Setup (please complete the following information):

OS: Azure Kubernetes Service
IDE: N/A
Library/Libraries: com.azure:azure-messaging-servicebus:7.15.2, com.azure:azure-cosmos:4.57.0, azureSdkBom 1.2.22
Java version: 17
App Server/Environment: Reactor/Netty
Frameworks: Spring Boot 3.2

If you suspect a dependency version mismatch (e.g. you see NoClassDefFoundError, NoSuchMethodError or similar), please check out Troubleshoot dependency version conflict article first. If it doesn't provide solution for the problem, please provide:

verbose dependency tree (mvn dependency:tree -Dverbose)
exception message, full stack trace, and any available logs

Additional context
Add any other context about the problem here.
Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[x] Bug Description Added
[x] Repro Steps Added
[x] Setup information Added

We found the issue.  It was due to our client code not returning and this caused the lock messages.  So I'm closing the issue.


boards: arm: Fix clock frequency for RP2040 boards

Adjust the clock frequency for RP2040 boards to be 125MHz as is found on most of them.

Seems to work for me on a Xiao RP2040


Implement basic unit testing

I propose we aim for a first 1.0 version which is released on pypi and aims for users to use the CLI resulting either in just parquet files (using the pagination feature) or full-on GBQ.
As good practice (and good exercise), implement unit testing using pytest following the guidelines from Real Python:

tutorial on general testing
tutorial on pytest
Martin Fowler's test pyramid
DORA: Test Data Management

Scope:

support Python 3.6+ (using tox); we choose 3.6 as minimal version because we use f-strings (see this overview of Python versions)
support coverage with pytest-cov, aim for 90%+ code coverage
setup CI/CD on GitHub, such that new versions to master get pushed to pypi

Starting point:
pytest --cov=statline_bq tests/
================================================================= test session starts ==================================================================
platform darwin -- Python 3.8.2, pytest-5.4.3, py-1.10.0, pluggy-0.13.1
rootdir: /Users/dkapitan/git/dataverbinders/statline-bq
plugins: cov-2.11.1
collected 1 item

tests/test_statline_bq.py .                                                                                [100%]

---------- coverage: platform darwin, python 3.8.2-final-0 -----------
Name                      Stmts   Miss  Cover
---------------------------------------------
statline_bq/__init__.py       1      0   100%
statline_bq/cli.py           33     33     0%
statline_bq/config.py        19     19     0%
statline_bq/utils.py        388    388     0%
---------------------------------------------
TOTAL                       441    440     1%

@galamit86
I got up to 18%, but from here it is a lot of testing how the code interacts with GCP. Not sure what's the best way to go about this. Feels a bit overkill to test wrapper functions that we've written for the GCP Python clients.
The parts where logic is processed, e.g. looking up the latest version, seems worth writing tests for but not sure how to do that. I think that no longer qualifies as a unit test, but it actually is an integration test.
Perhaps I was a bit too optimistic. For now at least we have setup some basic testing. What do you think?

We could use this fake-gcs-server

@dkapitan

I'm happy to add a separation between "user-API" and non-user things, to #73, after that is done we can come back here and discuss best test strategies.
I also agree that most of our more relevant tests seem to be integration tests rather than unit tests - I think that is because we mostly curate and assemble existing functions and methods - native python, google apis, etc.
For the GCP - we could use an emulator - but if we use an actual GCP environment - why would we need to tear it down each time? What's wrong with creating, populating and maintaining an environment that exists for testing purposes only? I thought about having a single project (so we won't have the dev, test and prod division), with a couple of folders on GCS and a couple of datasets on BQ.

Sounds like a plan: separate public from private methods and then setup an GCP environment for integration testing.

@galamit86
now that the refactoring is done, could you take it up from here with the following:

[ ] integrate this branch with refactored code
[ ] merge into master, so the minimal unit test setup is integrated
[ ] setup integration test on dataverbinders-test project

[ ] test GCS upload function

@dkapitan
This is a good opportunity to be explicit about our git/github workflow, and maybe fix some things I might be doing wrong. As it stands, to integrate our work and continue working, I would:

Pull your work (branch origin/issue-73-implement-testing) to a new local branch on my machine.
Rebase it onto my local master (which already includes the refactoring, and is of course the same as origin/master)
Make any changes needed to make sure it runs correctly (none needed in this instance).
Push it back to origin (origin/issue-73-implement-testing), using push --force
Create a PR in github, and use "rebase & merge" to merge origin/issue-73-implement-testing onto origin/master.
Pull origin/master locally
Delete issue-73-implement-testing from origin and locally.
Create a new branch locally to continue work.

That last part feels strange (having to delete and recreate branches), but I haven't figure out a better way yet - as far as I understand it, rebasing creates similar-but-different commits, requiring push --force, and making the deletion necessary. Do you have a better way in mind?

@galamit86
Sounds good to me. Deleting it seems OK. As I understand it, strictly speaking a ticket/issue should have a non-changing scope. Because we don't adhere to that, we re-create branches as we go along. No tension in any case.

@dkapitan
Great.
One more question - in the interest of keeping a tidy history, are you using "fixup" to remove your WIP commit, effectively merging it with the commit that came before it?

@galamit86
More than ok, is definitely a lot cleaner

@dkapitan
What do you think of this implementation for an integration test?
There are a couple of specific issues (below) I'm not certain of, but also wondering if the general setup looks proper to you.

I currently store the "true" parquet files to match against in tests/data/SOME_ID. I've only pushed one of them (83585NED), and all 4 datasets together are about 20MB. Is there a way to place them elsewhere? Should we look at Git large file storage for this?
If the source data is updated, this test will fail - since the new file will be different. Do you think we should use the checking last_modified that we do in _skip_dataset beforehand, and indicate that somehow?
Another thing that might happen, is when a dataset is updated from v3 to v4. main will automatically recognise this, and process the v4 dataset. This will also cause a failure. Again, I can use _check_v4 to check the version matches, and indicate that.

@galamit
Storing 20MB of files is fine, we don't need LFS for now.
If would elaborate the test a bit more, indeed to check new version within v3, and also with _check_v4
Finally, I would put the configuration of the actual GCP test project in config.toml instead of hardcoding it. Or a separate toml in tests is fine by me, too

@dkapitan
Updated code, adding two assertions:

new_modified == test_modified checking the metadata modified field
odata_version == ODATA_VERSION checking whether the dataset was updated from v3 to v4.

I'm also reading the config from file: CONFIG = config.get_config("statline_bq/config.toml") - meaning I use the actual config file that we use for the library, and then I dot into the test part: CONFIG.gcp.test.project_id.
Finally, something strange happened when running the tests just now. One of the datasets, for 84799NED failed, and the reason is that the metadata file I downloaded 3 days ago, and I use as truth is slightly different that the one I get when downloading it now. The older one has "ID": 936, while the new one has "ID": 937. No other fields are different, including "Updated", "Modified" or "MetaDataModified".
I am not sure how to address this. My feeling is I it's worth spending more time digging into this trying to understand what exactly happened, so I want to update update the truth files, and move on. What do you think? Maybe this also means we should introduce this "skip" into the actual test, and not compare the full metadata file?

@galamit86
agree: let's just update the truth files for now

@dkapitan
Same issue happened again - ID field updated from 937 to 939 in the metadata.
The information is taken from the CBS catalog here. I have not found official documentation saying so, but it seems clear to me that the ID is not relevant, and can be ignored for the purpose of validating the data. I plan to add a specific skip in the test, to avoid the tests from unnecessarily failing constantly.
Let me know if you have an objection or a different perspective.

No objection


Set a consistent window location

Whenever I start up mousejiggler, I have to hunt for where the window ends up. This makes a consistent location.
I'd prefer putting it in the top right corner of the primary display, but that's more than a single line and is harder to test.

I just noticed that much of this functionality has been handled on your dev branch. I'll rebase and see if any work needs to be done from there.


WebGL hit detection transform does not take pixel ratio into account

When trying WebGL hit detection with the icon-sprite-webgl.html example on a Retina display (pixel ratio: 2), I noticed that hovered icons and hit detections do not align. With a pixel ratio of 1, hit detection works as expected.

This probably has to do with the fact we're not looking up the right pixel here:
https://github.com/openlayers/openlayers/blob/32d1d9214cf8510b9b9870955eec2c4800c3d205/src/ol/renderer/webgl/PointsLayer.js#L545
Will try to look into this ASAP. Thanks for the issue


Restyling/install screen

Redesigned install screens.

Noting that I made a review pass on the Java portions a few weeks.
Thanks for updating the PR based on my comments, looks good!


ghcup list - show available ghc versions?

I teach Haskell to newbies, they won't know what version to put into ghcup install <VERSION> could there be a list command?

Interesting suggestion. Would that be a static list or would it involve scraping upstream urls? Static list sounds easy enough and would also indicate "supported" versions.

@hasufell if you can think of a general text format that's happens to be easy to parse in bash (probably a tab-separated-values format would be easiest? any other options?) we could host such a meta-file on downloads.haskell.org

@hvr great suggestion. I wonder if it would make sense to even use signify to prepare a release folder for GHC versions. See LibreSSL for example: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/
And this is the format of the signify file: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/SHA256
Should be pretty easy to handle with awk. In addition, you could provide a symlink url, which points to the "best guess" tarball (latest version built on debian for example), so the script can just fetch https://downloads.haskell.org/~ghc/ghc-latest-unknown-linux.tar.xz (which still has a ghc-<version> subfolder, so the script can detect what version we actually fetched).
Just throwing out ideas here.

https://github.com/haskell/ghcup/issues/21


Remove to/from/as_raw_hash functions

Done as work towards #2881 (first bullet point).
As stated in the issue the goal is:
"An API where only "general hashes" (i.e. hashes defined by their algorithms) can hash arbitrary data, while "normal hashes" (e.g. Txid) can only be constructed from actual transaction data, from deserialization, and from "hash casting" by converting to/from byte arrays."
As such, we should not provide getters and setters for raw Hash type.

ACK the code but I'd prefer to modify the commit message to not be misleading.

ACK the code but I'd prefer to modify the commit message to not be misleading.

I went to do this but I don't see which bit is you think is misleading sorry. A added as to the list of functions in the commit brief log and mentioned the unit test - but I don't think this is what you meant. Can you check and tell me please?

Force push is changes to the commit log only, no code changes.

What I found misleading is the claim that this has anything to do with hash newtypes not being general hashes. A Txid is a sha256 and should be convertible to general hash directly. Just like NonZeroUsize is a usize and can be converted to it.
The reason to still remove this is to not produce unstable API from the hash_newtype macro while hashes is unstable because crates that want to be stable soon need to use the macro.
So the message should state this.

I could be convinced either way (and it's fine to change the commit message to say "we're doing it to shrink the API to things we agree on", but I also had no idea what you meant about the original message being misleading. There is a comment thread above where you stated your case and I disagreed and there is no clear conclusion. And then the request to change the message was in a separate thread so it wasn't clear it was related.
I'm often guilty of this too -- but it's worth the effort to slow down and make your position clear rather than saying things are "wrong" in an unspecified way.

I guess the confusion often stems from GH not allowing commenting on commit messages. Another thing is that this PR is weird because we all want it for completely different reasons and simultaneously I think the reason stated in the message doesn't make sense.

this PR is weird because we all want it for completely different reasons and simultaneously I think the reason

lol, now that is rough consensus

Force push is commit log change (copied to PR description) only, no code changes.


Fixed Giveaway start Hosted By Bug

How do I participate?

????
On Sat, Feb 19, 2022 at 12:17 PM PixeltonPi @.***>
wrote:

the file is in file changed
—
Reply to this email directly, view it on GitHub
https://github.com/saiteja-madha/discord-js-bot/pull/100#issuecomment-1045897311,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AX2ARGWPOEBRW5MWCU2LAS3U344IHANCNFSM5O2BWKQA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID:
@.***>


after installation, "source ctypes.sh" will work from anywhere?

why do I need not to specify 'source /usr/local/bin/ctypes.sh' anymore? 'source ctypes.sh' works directly, how does source find it, I don't think 'source' will search PATH

source actually does use PATH! 😃


Add Deflector Shields

Create a Deflector Shield part that uses EC to operate ... Some form of armored umbrella that can be extended and retracted as well as having an invisible texture (specular?)
Extended uses EC and retracted uses none

Created DCK Shields ... closing


UniqueConstraint with InlinePanel

Issue Summary
Converting a unique_together to UniqueConstraint results in an uncaught IntegrityError when trying to add a duplicate in a related model with InlinePanel.
    class Meta:
        # uncaught IntegrityError
        constraints = (
            models.UniqueConstraint(fields=["short_name", "meeting_site"], name="unique_room"),
        )
        # this works
        unique_together = ("short_name", "meeting_site")

I found this older issue relating to unique_together: https://github.com/wagtail/wagtail/issues/2136
I also found this issue, but it seems it's only for images? https://github.com/wagtail/wagtail/issues/8715
Technical details

Python version: Python 3.12.7
Django version: 5.1.2
Wagtail version: 6.2.2
Browser version: FF 131.0.2

This likely is an issue in django-modelcluster. The fix might be to pass include_meta_constrains=True to _get_unique_checks() in https://github.com/wagtail/django-modelcluster/blob/7e53fcee2fac812dae04b89da37ec3cc460cfb5a/modelcluster/forms.py#L137.
(Django ref: https://github.com/django/django/blob/97c05a64ca87253e9789ebaab4b6d20a1b2370cf/django/db/models/base.py#L1404)

This seems to fix it, thanks for the very quick response.


fix: cardano-services postgres ssl option resolution

https://input-output-rnd.slack.com/archives/C01S2DD5B0D/p1686064086954539?thread_ts=1686056770.853969&cid=C01S2DD5B0D
Currently testing this out

Seems to work as expected. Just fixed the prettier issue.
Ready to be merged

The e2e failure is unrelated


add terminate_delay and fork_delay parameters to track_process

These seem to be available from keepalived 2.0.16 onwards. Which also changes purpose of the delay parameter to be the default for both.

REFERENCE.md is outdated

Please update REFERENCE.md with
$ bundle exec rake strings:generate:reference

Add it to the PR and CI should continue.


tests/tt_eager/python_api_testing/unit_testing/misc/test_bert_ops.py::test_bert_linear hangs occasionally

Both recorded runs occurred on vm56:
https://github.com/tenstorrent/tt-metal/actions/runs/9653205303/job/26625199314
https://github.com/tenstorrent/tt-metal/actions/runs/9600295478/job/26476597759
Should verify if latest FW drop fixes this issue

doesn't occur anymore with latest FW


Where is the list of breaking changes for the version 9?

Does this list exists?

Yes, it's in the changelog. https://github.com/isaacs/node-glob/blob/main/changelog.md#90


get() returned more than one Tag -- it returned 2!

Hello
Sometimes when adding tags to an existing object, I am getting the following error.
get() returned more than one Tag -- it returned 2!

Here is the code I am using
...
publication.tags.set(all_tags, clear=True)
publication.save()
# I also tried
publication.tags.set(all_tags)
publication.save()
# and
publication.tags.add(*all_tags)
publication.save()
# but still getting the same issue
...

Notes:

I am not using any get function
Django version 4.1

@naskio do you have a longer stack trace? please post more details.

2022-11-16 01:06:22.022 | ERROR    | apps.data_collection.management.commands.tagging:handle:69 - get() returned more than one Tag -- it returned 2!
Traceback (most recent call last):

> File "/app/apps/data_collection/management/commands/tagging.py", line 63, in handle
    publication.tags.set(all_tags, clear=True)
    │           │        └ ['libération', 'Direction', 'Algérie Presse Service', 'guerre de libération', 'EN', 'Communauté nationale', 'Vidéo', 'Abdelma...
    │           └ <taggit.managers.TaggableManager: tags>
    └ <Publication: La communauté algérienne en Libye commémore le 68e anniversaire du déclenchement de la Glorieuse guerre de libé...

  File "/usr/local/lib/python3.9/site-packages/taggit/utils.py", line 124, in inner
    return func(self, *args, **kwargs)
           │    │      │       └ {'clear': True}
           │    │      └ (['libération', 'Direction', 'Algérie Presse Service', 'guerre de libération', 'EN', 'Communauté nationale', 'Vidéo', 'Abdelm...
           │    └ <taggit.managers._TaggableManager object at 0x7ff903015be0>
           └ <function _TaggableManager.set at 0x7ff90e52dee0>
  File "/usr/local/lib/python3.9/site-packages/taggit/managers.py", line 273, in set
    self.add(*tags, **kwargs)
    │    │    │       └ {}
    │    │    └ ['libération', 'Direction', 'Algérie Presse Service', 'guerre de libération', 'EN', 'Communauté nationale', 'Vidéo', 'Abdelma...
    │    └ <function _TaggableManager.add at 0x7ff90e52db80>
    └ <taggit.managers._TaggableManager object at 0x7ff903015be0>
  File "/usr/local/lib/python3.9/site-packages/taggit/utils.py", line 124, in inner
    return func(self, *args, **kwargs)
           │    │      │       └ {}
           │    │      └ ('libération', 'Direction', 'Algérie Presse Service', 'guerre de libération', 'EN', 'Communauté nationale', 'Vidéo', 'Abdelma...
           │    └ <taggit.managers._TaggableManager object at 0x7ff903015be0>
           └ <function _TaggableManager.add at 0x7ff90e52daf0>
  File "/usr/local/lib/python3.9/site-packages/taggit/managers.py", line 152, in add
    tag_objs = self._to_tag_model_instances(tags, tag_kwargs)
               │    │                       │     └ {}
               │    │                       └ ('libération', 'Direction', 'Algérie Presse Service', 'guerre de libération', 'EN', 'Communauté nationale', 'Vidéo', 'Abdelma...
               │    └ <function _TaggableManager._to_tag_model_instances at 0x7ff90e52dc10>
               └ <taggit.managers._TaggableManager object at 0x7ff903015be0>
  File "/usr/local/lib/python3.9/site-packages/taggit/managers.py", line 223, in _to_tag_model_instances
    tag = manager.get(name__iexact=name)
          │       │                └ 'UN'
          │       └ <function QuerySet.get at 0x7ff910d86f70>
          └ <QuerySet [<Tag: South Korea>, <Tag: India>, <Tag: Canadian>, <Tag: Australia>, <Tag: Barça>, <Tag: Xavi>, <Tag: Ukraine>, <T...
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 499, in get
    raise self.model.MultipleObjectsReturned(
          │    │     └ <class 'taggit.models.Tag.MultipleObjectsReturned'>
          │    └ <class 'taggit.models.Tag'>
          └ <QuerySet [<Tag: South Korea>, <Tag: India>, <Tag: Canadian>, <Tag: Australia>, <Tag: Barça>, <Tag: Xavi>, <Tag: Ukraine>, <T...

taggit.models.Tag.MultipleObjectsReturned: get() returned more than one Tag -- it returned 2!

NOTE:
At the begging I inserted many tags without having TAGGIT_CASE_INSENSITIVE = True but I added it later.
so I guess that I may have inserted some tags (for example UKRAINE and Ukraine) which were not the same but became equal after enabling case insensitivity (having duplicates in the DB).

@naskio thank you very much for that detail! It sounds very important.
If in your system you have that problem, the easiest thing to do would be to normalize manually in a database migration. So for example you find all of the TaggedItems that are pointing to UKRAINE and then point them to Ukraine. From there, you should be able to delete UKRAINE.
You'll need to do some database cleanup. I think though that this error should have its own handling, though. And it might even make sense for us to write a helper management command... but at the very least we should document what to do.

Good first issue: catching MultipleObjectsReturned and rethrowing it with a better error message to handle this.

@rtpg
Thanks for your answer.
Yes, I thought of this solution. it makes sens to have command for this.

What's the status on this? Up for grabs?

@AmaMidzu it's yours for the taking! Going to assign you for now just so we can keep track

Hello!
I'm new to contributing to OSS.
Can I take this up?

bump @rtpg

@naskio I've figured out the first part of your recommendation. For the management command to remove duplicate tags, what do you think should be the preference?

Remove all duplicates except the first one.
Keep only the uppercase version, like "UKRAINE" in the example.
If there is no uppercase version, on what basis should we remove the other tags and keep a single tag?
also what would be a suitable name for the management command? "remove_duplicate_tags" or "deduplicate_tags" or something else?

Hi @fazeelghafoor

I suggest we keep the first tag and merge all duplicates into it (relationships), then delete the duplicates.
The name deduplicate_tags sounds appropriate to me.


Perform a README overhall for easier understandability

TODO:
1. Refactoring

[x] a. Move all the detailed instructions into an advanced section.
[x] b. Promote one-liners to the beginning of the Readme.
[x] c. Add the logo.
[ ] d. ~Add the cloning instructions into the one-liners so the new users clone the stable branch by default.~ <- No longer needed. It's been made the default.
[x] e. Use the correct copyright notice

2. Langauge additions:

[ ] a. French
[ ] b. Spanish
[ ] c. Hindi
[ ] d. Chinese - modifications after 1.

1{a., b., c., e.} will be taken care of by PR #225


galleries - refactor to make them consistent

we have some inconsistencies in our gallery treatments, especially on
campaign confirmation page:

/campaigns:

Fixed with Neue 6.0 cleanup.


Problem with https://github.com/golang-sql/sqlexp package.

Could you please take a look into this line -> https://github.com/denisenkom/go-mssqldb/blob/1598eaf61e43a1bb1b66231abe49f8613b544d64/token.go#L790. Right now because of incompatible changes here -> https://github.com/golang-sql/sqlexp/commit/d42ba59f5c11011b1df5abc536ee75821c22bdac go-mssqldb packages just sopped working with message:
#5 12.59 pkg/mod/github.com/denisenkom/go-mssqldb@v0.12.0/token.go:790:70: cannot use info.Message (type string) as type fmt.Stringer in field value:
#5 12.59 	string does not implement fmt.Stringer (missing String method)

so you probably need to implement Stringer interface.

Pull latest.
On Thu, May 19, 2022, 13:32 Dmitry Suhinin @.***> wrote:

Could you please take a look into this line ->
https://github.com/denisenkom/go-mssqldb/blob/1598eaf61e43a1bb1b66231abe49f8613b544d64/token.go#L790.
Right now because of incompatible changes here -> golang-sql/sqlexp@
d42ba59
https://github.com/golang-sql/sqlexp/commit/d42ba59f5c11011b1df5abc536ee75821c22bdac
go-mssqldb packages just sopped working with message:
#5 12.59 @.***/token.go:790:70: cannot use info.Message (type string) as type fmt.Stringer in field value:
#5 12.59 	string does not implement fmt.Stringer (missing String method)
so you probably need to implement Stringer interface.
—
Reply to this email directly, view it on GitHub
https://github.com/denisenkom/go-mssqldb/issues/750, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAFYLMJMWRDRB76LZD2M4B3VK2CKTANCNFSM5WNCQ6SA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


Exception: You need to install rdkit and its dependencies.

Hi everyone,
So, I have been taking this error:
Exception: You need to install rdkit and its dependencies.
When I go to the source of the problem:
\MolScore\molscore\utils\gypsum_dl\Steps\IO\Web2DOutput
Line 25-31:
try:
    from rdkit.Chem import rdDepictor
    from rdkit.Chem.Draw import rdMolDraw2D
    from rdkit.Chem.Draw import PrepareMolForDrawing
    from rdkit import Chem
except:
    Utils.exception("You need to install rdkit and its dependencies.")

So, I tried to import modules one by one. Problem was about rdkit.Chem.Draw giving this error:
>>> from rdkit.Chem.Draw import rdMolDraw2D
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/conda/envs/molscore/lib/python3.9/site-packages/rdkit/Chem/Draw/__init__.py", line 21, in <module>
    from rdkit.Chem.Draw import rdMolDraw2D
ImportError: libXau.so.6: cannot open shared object file: No such file or directory

To solve this error, I needed to change environment with the following:
apt update && apt install -y libsm6 libxext6
I am not sure why this was happening, but I wanted to share it since it might be helpful to others experiencing the same error or for the contributors to check the code.


Hi All,Could we load already generated las file in carla simulator

We don't have support for LAS format at the moment.


feat(pdk): add authentication context

Summary
Introduces authentication context.
For the authentication plugins that not only require providing the key(password) but also a username, like basic-auth, Kong set the given_username into ngx.ctx.authentication_context. A typical use case is that users can log or inject the given_username into http log event when key(password) and username are invalid.
For example:
plugins:
- name: basic-auth
- name: http-log
  config:
    http_endpoint: ""
    method: "POST"
    custom_fields_by_lua:
      authentication_context.username: "return kong.client.get_authentication_context() and kong.client.get_authentication_context().username"

Checklist

[x] The Pull Request has tests
[ ] There's an entry in the CHANGELOG
[ ] There is a user-facing docs PR against https://github.com/Kong/docs.konghq.com - PUT DOCS PR HERE

Issue reference
FTI-5281

Hi @vm-001, we looked this together with @hanshuebner. To recap what discussed.
Current State

kong sets ngx.ctx.authenticated_consumer to Consumer entity on successful authentication (either by credential to Consumer mapping or perhaps by OIDC Claim / SAML Attribute mapping to Kong Consumer).
kong sets ngx.ctx.authenticated_credential to credential (of varying types) or in case of say OpenID Connect it just sets it to { id = tostring(credential_value) } (where credential_value is value found with credential_claim which defaults to standard sub). Again this happens on successful authentication only.
Plugins can set ngx.ctx.authenticated_groups too.
Plugins usually also set headers for these X-Consumer-ID, X-Consumer-Username, X-Consumer-Custom-ID, X-Credential-Identifier, X-Authenticated-Groups, X-Consumer-Groups.
Usually the plugins also remove the credentials given from the request (the hide_credentials config).

The PR Proposal
The PR here is about storing the user given credentials. The problem with Kong Credentials and User given credentials is that they are not standardized. E.g. in this PR we see this data added to context:
{
   username = "...",
}

Well what if there is key auth? Do we really want to set:
{
   key = "valid or invalid key",
}

Or OIDC:
{
   access_token = "<jwt>",
}

And then log the valid and invalid credentials or part of them? It may also be problematic on privacy side and this can create a foot gun where things like email addresses get logged and collected to places where they should not be logged.
I think the idea is fine, but I think we need to think this further. We are not too happy how it currently works, especially the ngx.ctx.authenticated_credential and how it gets used to run authentication plugins in OR  fashion, and how the type of it is basically unknown to anything else than the one that sets it. The same unknown type problem is here with authentication context. It is hard to use by any other code, even the example:
authentication_context.username: "return kong.client.get_authentication_context() and kong.client.get_authentication_context().username"

Makes assumptions that there is username and when there can be many authentication plugins it gets more complicated. Thus I think we need to think this a bit more.
The PR makes changes to all the plugins, but to make it really usable across the board, we would need to make changes to perhaps all the authentication plugins.
Further thinking is it possible to standardize authentication context:
{
   identifier: my-username
   method: basic-auth
}

{
   identifier: hash of give key?
   method: key-auth
}

{
   identifier: <client-id|username|opaque-token|jwt-token|auth-code...>
   method: openid-connect
}

When it is ok to make these available, what are the use cases, should they be filtered / sanitized before say logging them or even before putting them in context. Thus we want to leave it on a table to let this idea mature a bit.
@hanshuebner, do you have something to add?

I think the key point of this PR is that the credential information that was used in an authentication attempt is stored in the authentication_context before the actual authentication runs.  This ensure that the information is available in the log phase, even if the authentication fails.  For many use cases, it would also be possible to extract the authentication information from the request header in the log phase.  I think for the specific use case that is being addressed by this PR (and the FTI/A-HA tickets), the cost of the implementation (increased request context size and maintenance burden of the extra authentication_context, which only serves this specific purpose) is too high.
As @bungle described, our authentication related data structures and mechanisms are already somewhat difficult to understand and maintain.  We'd like to change that, but that is probably not going to happen very soon.  For the logging use case, I would suggest a simpler approach that does not require a change in how authentication is handled in Kong overall.

This PR is set for the 3.5.0 milestone but didn't receive updates in the recent past. @vm-001 please remove the milestone if you think this won't make it.

We discussed this PR in PR Queue Review. I'm removing it from the 3.5 milestone so we can design this feature better (with a design doc).


File yarn.lock points to incorrect URLs

The current yarn.lock file points to:
https://registry.npmjs.org/
Instead of:
https://testnet.thegraph.com/npm-registry/
This prevented me from building with yarn, until i removed and recreated this file. So it should be updated in the repo, or removed so that it's recreated by the user.

Resolved with #32


Try to unbreak backend-common dependencies

Fixes #25303
Does this in two ways:

Make techdocs-cli not depend on backend-common at all in the first place
Make backend-common have (hopefully all) the required dependencies that its cross-package imports imply that it needs

Thanks for this fix! Can you say approximately when this will be available?
(The bot message made me bit insecure, whether this might be released not until July 16 😬 )

Should be out in a patch release later today.


copy feature (make only the default x or y ordinate increase by 10 instead of all xyz coordinates increase)

Lets keep z and y the same and shift x by 10 as the default position of the copied cortical area.


Fetching channel types using API; updated type imports

Description
Currently the notification channel types are hard coded in frontend which results in missing new channel types added by backend and thus those channels don't get added to the dropdown options for setting up alert notifications.
Issues Resolved
#903
Check List

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

The changes LGTM. @amsiglan explained offline that he would investigate test failures.

All tests passing locally. Lately github CIs are getting canceled midway. Need to investigate that, but we are good to merge this PR.


extract reader/writer impl, increase their default buffer and minimize footprint when possible.

Coverage increased (+0.03%) to 58.69% when pulling 8fec7f340c50b2d53eb4e79ab536b309159060d4 on aozarov:temp2 into 05e3f0a856850813b7a07462dc2e4ece3ef5e4ce on GoogleCloudPlatform:master.

Coverage increased (+0.03%) to 58.69% when pulling 8fec7f340c50b2d53eb4e79ab536b309159060d4 on aozarov:temp2 into 05e3f0a856850813b7a07462dc2e4ece3ef5e4ce on GoogleCloudPlatform:master.

Coverage increased (+0.03%) to 60.89% when pulling b9b43831818a24927e453d5cde7987a96dca71ac on aozarov:temp2 into e0c03c91b64b55fe9c8612c85bbc18ec52cba37a on GoogleCloudPlatform:master.

Coverage increased (+0.03%) to 60.89% when pulling b9b43831818a24927e453d5cde7987a96dca71ac on aozarov:temp2 into e0c03c91b64b55fe9c8612c85bbc18ec52cba37a on GoogleCloudPlatform:master.

LGTM

Coverage increased (+0.04%) to 68.33% when pulling 3c9129935ddaf1d026d3781d7cb3d3f805d9cc0f on aozarov:temp2 into c120b6801d20be1730c6945368a5c47a85711764 on GoogleCloudPlatform:master.