Contract Internal Audit / Refactor

Closes: #XXX
What is the purpose of the change

Add a description of the overall background and high level changes that this PR introduces

Brief Changelog

Lets us know what changes you made

Testing and Verifying
(Please pick one of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added unit test that validates ...
Added integration tests for end-to-end deployment with ...
Extended integration test for ...
Manually verified the change by ...

R4M, merging by tonight if no one has immediate outstanding objections in favor of new PR on contract tests.

Can you implement a getter for Dealsthat just returns the contents of the Struct? Json RPC's have trouble reading structs externally.

Getter implemented


Roll engine 471a2c89a689..3661d5e43b83 (1 commits)

https://github.com/flutter/engine/compare/471a2c89a689...3661d5e43b83
git log 471a2c89a689ff1d5c70a5deb4565f646380bce6..3661d5e43b83106498e3a4ce2226f976cebed021 --no-merges --oneline
3661d5e43 Re-land "Buffer lifecycle in WindowData" (flutter/engine#8032)
The AutoRoll server is located here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md
If the roll is causing failures, please contact the current sheriff (mklim@google.com), and stop
the roller if necessary.

Trybots failed. These were the failed builds: https://cirrus-ci.com/task/4872380554936320

AutoRoller is stopped; closing the active roll.


chore: Use transaction_session_logger() only where necessary. Rename/refactor as handle_skipped_award() #249

@yolile A test is failing, because in the past we just logged at exception level and let all these errors pass through. (In production, this was more or less fine, as we'd see it in Sentry.)
Anyhow, this might be a case of the test fixtures being incorrectly configured – I haven't looked into it.

Anyhow, this might be a case of the test fixtures being incorrectly configured – I haven't looked into it.

Yes, the current tests used a list of contracts but then the same award, borrower, etc for all of them, causing duplications. This is not how the endpoint works in reality, so I just changed the contracts mock data to one contract instead of 5 and synchronized the ids, etc of the other mock files to align.
I don't think we were testing multiple contracts anyway, but if we want to do that, we might want to change our mock data or how we do the mock.

Aha, thanks for explaining the resolution. Up to you then to merge :)


Huge memory overhead per table/DB in memory

Thanks for this great library!
I've been trying to use it to do some in-memory caching, but find out when having many tables/dbs, the overhead per db/table is quite big.
Here's a simple test program:
const Sqlite = require("better-sqlite3");
const fs = require("fs");

function sleep(n) {
  return new Promise((resolve) => setTimeout(resolve, n));
}

const globalDb = new Sqlite(":memory:");
globalDb.pragma("journal_mode = OFF");

async function runSqlite(index, nRows) {
  // const db = globalDb;
  const db = new Sqlite(":memory:");

  const tableName = `map${index}`;
  db.prepare(
    `CREATE TABLE ${tableName} (key TEXT PRIMARY KEY, value INTEGER)`
  ).run();

  await sleep(5000);

  db.prepare(`DROP TABLE ${tableName}`).run();
  db.close();
}

async function main() {
  const statFile = fs.openSync("sqlite-mem.tsv", "w");
  fs.writeFileSync(statFile, "rss\theap_total\theap_used\n");

  for (let i = 0; i < 1000; i++) {
    console.log(i);
    if (i % 1 === 0) {
      console.log("iter", i);
      const stat = process.memoryUsage();
      const row = [stat.rss, stat.heapTotal, stat.heapUsed].map((num) =>
        (num / 1024 / 1024).toFixed(2)
      );
      fs.writeFileSync(statFile, row.join("\t") + "\n");
    }

    const promises = [];
    // 10000 tables: 1.4g
    // 10000 dbs: 1.6g
    for (let inner = 0; inner < 10000; inner++) {
      promises.push(runSqlite(inner, 0));
    }

    await Promise.all(promises);
  }

  fs.closeSync(statFile);
}

main();

The test shows, with

10000 empty tables: RSS usage is 1.4GB, that's 140kb per empty table
10000 empty DBs: RSS usage is 1.4GB, that's 140kb per empty DB
10000 DBs with a empty in it: RSS usage is 1.6GB, that's 160kb per DB with a empty table

This overhead seems a bit off, is there anything obvious I'm doing wrong here? Thanks again!

Are you sure, that you are measuring / interpreting the sizes correctly? After having quick look on the code, you are measuring the heap size of the whole process including JavaScript objects and eventual bound C++ objects. Due to how NodeJS handels the mapping of JavaScript <> C++ there will always be unavoidable overhead when using libraries written in C++. In the end you are measuring the size of a database instance, required structures for mapping them to the JavaScript context, the created scopes / contexts from the Promises and so on.
Without having dug to deep into the sqlite c++ code, I would say that 140kB ~160kB for an empty, ready to go in-memory database containing one table is a pretty reasonable size. A database is not just a bit of schema definition. Data structures, pointers etc. have to be allocated and hold in memory - even if there is nothing stored or happening.
I the end this minimal size is part of the tradeoff of having 10.000 independent databases instead of one shared instance.

Thanks for the reply. My code also includes a way to experiment with 1 DB + 10000 tables, the result is similar, 140kb overhead PER TABLE, which sounds a bit off to me.
My use case is to use sqlite as an in-memory cache table to get around the fact that JS does not provide efficient data structures that supports sorting (with multiple keys too), and I need multiple tables (or DBs whichever works), because all these cache objects have slightly different schemas (on the sort key).

My code also includes a way to experiment with 1 DB + 10000 tables, the result is similar, 140kb overhead PER TABLE, which sounds a bit off to me.

Cannot reproduce.
Here's a simplified version showing that each DB does indeed have an overhead without any tables:
const Sqlite = require('better-sqlite3');

function sleep(n) {
  return new Promise((resolve) => setTimeout(resolve, n));
}

async function runSqlite(index, nRows) {
  const db = new Sqlite(':memory:');
  await sleep(5000);
  db.close();
}

async function main() {
  for (let i = 0; i < 1000; i++) {
    console.log('iter', i);
    const stat = process.memoryUsage();
    const row = [stat.rss, stat.heapTotal, stat.heapUsed].map((num) => (num / 1024 / 1024).toFixed(2));
    console.log(row);

    const promises = [];

    for (let inner = 0; inner < 10000; inner++) {
      promises.push(runSqlite(inner, 0));
    }

    await Promise.all(promises);
  }
}

main();

I get [ '709.79', '30.11', '13.13' ] for the second iteration, which makes it roughly 69kb per database. Entirely reasonable given how many objects each instance takes with all the methods it has (on better-sqlite3's end) and everything SQLite itself needs to set up.
Here's a single global database with 10k tables:
const Sqlite = require('better-sqlite3');

function sleep(n) {
  return new Promise((resolve) => setTimeout(resolve, n));
}

const globalDb = new Sqlite(':memory:');

async function runSqlite(index, nRows) {
  const tableName = `map${index}`;
  globalDb.prepare(`CREATE TABLE ${tableName} (key TEXT PRIMARY KEY, value INTEGER)`).run();

  await sleep(5000);

  globalDb.prepare(`DROP TABLE ${tableName}`).run();
}

async function main() {
  for (let i = 0; i < 1000; i++) {
    console.log('iter', i);
    const stat = process.memoryUsage();
    const row = [stat.rss, stat.heapTotal, stat.heapUsed].map((num) => (num / 1024 / 1024).toFixed(2));
    console.log(row);

    const promises = [];

    for (let inner = 0; inner < 10000; inner++) {
      promises.push(runSqlite(inner, 0));
    }

    await Promise.all(promises);
  }
}

main();

I get [ '168.81', '15.62', '13.86' ] for the second iteration, which makes it roughly 14kb per table. Also sounds pretty reasonable to me.
Here's a version of that where I don't run() the statement. Just to see how much memory the prepared statement takes before it can be garbage collected when the sleep finished and the scope is left.
const Sqlite = require('better-sqlite3');

function sleep(n) {
  return new Promise((resolve) => setTimeout(resolve, n));
}

const globalDb = new Sqlite(':memory:');

async function runSqlite(index, nRows) {
  const tableName = `map${index}`;
  globalDb.prepare(`CREATE TABLE ${tableName} (key TEXT PRIMARY KEY, value INTEGER)`); //<======= look ma, no run()!

  await sleep(5000);
}

async function main() {
  for (let i = 0; i < 1000; i++) {
    console.log('iter', i);
    const stat = process.memoryUsage();
    const row = [stat.rss, stat.heapTotal, stat.heapUsed].map((num) => (num / 1024 / 1024).toFixed(2));
    console.log(row);

    const promises = [];

    for (let inner = 0; inner < 10000; inner++) {
      promises.push(runSqlite(inner, 0));
    }

    await Promise.all(promises);
  }
}

main();

I get [ '62.63', '29.87', '12.72' ] for the second iteration, which means the prepared statements take roughly 3kb. Doesn't sound like that much to me, given that this holds things like the compiled byte code for the statement in memory, among other things.
Last thing: global database with 10k tables after everything has been garbage collected (this is what your app would actually do, first setup everything and then use the tables).
const Sqlite = require('better-sqlite3');

function sleep(n) {
  return new Promise((resolve) => setTimeout(resolve, n));
}

const globalDb = new Sqlite(':memory:');

function runSqlite(index, nRows) {
  const tableName = `map${index}`;
  globalDb.prepare(`CREATE TABLE ${tableName} (key TEXT PRIMARY KEY, value INTEGER)`).run();
}

async function main() {
  console.log('iter', 0);
  let stat = process.memoryUsage();
  let row = [stat.rss, stat.heapTotal, stat.heapUsed].map((num) => (num / 1024 / 1024).toFixed(2));
  console.log(row);

  const promises = [];

  for (let inner = 0; inner < 10000; inner++) {
    runSqlite(inner, 0);
  }

  // Give the GC some time to clean up the inside of runSqlite().
  await sleep(5000);

  console.log('iter', 1);
  stat = process.memoryUsage();
  row = [stat.rss, stat.heapTotal, stat.heapUsed].map((num) => (num / 1024 / 1024).toFixed(2));
  console.log(row);

  // Demonstrate that we actually have 10k tables in there
  console.log(globalDb.prepare("SELECT * FROM sqlite_master WHERE type='table'").all());
}

main();

This gives me a db with 10k tables at roughly 10kb per table (process takes 135MB, with empty Node.js taking 35 of that already).
Conclusion
When doing any sort of benchmarking, be careful what you measure. Most benchmarks are broken from the get go because of wrong assumptions.

Closing this unless a counter-example is posted


Image viewer in image grids

Hi! I'm using minimal theme for image grids feature mainly. And it would be great to have some kind of image viewer when you click on an thembnail image in grid. Currently when you press on an image it is shown enlarged, but only when you keep pressing mouse key. What i suggest is an image stays enlarged and you can scroll it to the next using arrow buttons (or scroll wheel) - some kind of viewer.

Not something that can be done via a theme. See previous issues. Needs to be requested on Obsidian app, see open feature requests in the official forum.


Project Build Error

Fairly new to maven, but I have pulled the code and have an error in my pom.xml fir for spring-cloud-gcp-data-spanner-sample.
Project build error: Non-resolvable parent POM for com.example:spring-cloud-gcp-data-spanner-sample:[unknown-version]: Could not find artifact org.springframework.cloud:spring-cloud-gcp-samples:pom:1.2.0.BUILD-SNAPSHOT and 'parent.relativePath' points at wrong local POM

What is the build command you're running and from which directory?

I am running mvn package in this directory:
spring-cloud-gcp-master\spring-cloud-gcp-samples\spring-cloud-gcp-data-spanner-sample

You'd want to first build from the root directory spring-cloud-gcp-master\.

Thanks for the replies, I'm very new to all of this, so I greatly appreciate any help you can provide. When I try to build from the root, I get a bunch of warnings, should I be worried & how can I fix them exactly? I don't clearly understand how to.. thanks!

No need to worry about warnings for your purposes. We do need to take care of them at some point though. You can also skip tests to make the build faster with mvn clean install -DskipTests.

Closing this one now. Please feel free to reopen if there is an error.


Add docs for hpx functions

Fixes https://github.com/STEllAR-GROUP/hpx/issues/985.
Proposed Changes
Documented few hpx future functions present in the ticket.
Any background context you want to provide?
GSoD'19 applicant.
@msimberg's review comment.

Please add license and copyright to all files (see https://94186-4455628-gh.circle-artifacts.com/0/hpx_inspect_report.html).

@hkaiser how does the docs looks now? do you think I can enhance anything else in this PR?

@tapaswenipathak there's an important part missing: the file you've added isn't automatically part of the generated documentation. You'd have to add it to the table of contents of another rst file (like this: https://raw.githubusercontent.com/STEllAR-GROUP/hpx/master/docs/sphinx/manual.rst). It would most likely fit best in or close to this section: https://stellar-group.github.io/hpx/docs/sphinx/branches/master/html/manual/writing_single_node_hpx_applications.html. You should preferably move the code samples into source files in the examples subdirectory and add them with literalinclude. This way we're sure that the code actually compiles and runs. Sorry for not making this clear earlier.

I'd suggest you move this to here: https://stellar-group.github.io/hpx/docs/sphinx/branches/master/html/manual/writing_single_node_hpx_applications.html. It could be called "Basics of asynchronous programming" or something in that direction. You'll also need to add the example code to our CMakeLists.txt here and create a new subdirectory for them with a CMakeLists.txt like this. Please build and run your code snippets to make sure that they are correct.

@msimberg: How does 4197cd4f0ddefc3ea09cd9cabb1ce18abe0c4e53 look?

@tapaswenipathak do you plan to work on this?

@msimberg I will implement the remaining things; sorry for the delay.

@msimberg I will implement the remaining things; sorry for the delay.

No worries. Thanks for working on this!

@tapaswenipathak please let us know if we can help you build HPX. That will be the easiest way to resolve the problems that the CI is reporting.

Closing this for now to avoid unnecessary CI builds. @tapaswenipathak this doesn't mean we're not interested in the content of the PR. Feel free to reopen if you get the time to continue this at some point in the future!

@dimitraka WOuld you be able to have a look what can (and should) be salvaged for our documentation?

@dimitraka WOuld you be able to have a look what can (and should) be salvaged for our documentation?

Sure, I will take a look and if necessary open a PR


fix Content-Length header values for /range endpoint

This supersedes #345 by fixing the value for the Content-Length for the /range endpoint.

Great, I'll go ahead and merge this then. Thanks @Lukasa!

Thanks, when do you usually deploy changes to httpbin.org ?

promoted to production!

Hi, this is still working as previously on httpbin.org (response sent at the end of duration, not streamed incrementally). Not sure if question should go here, but when this will be fixed in httpbin.org site as well?


How OSS companies changed in the last few years

Yesterday, I had some discussions with another open-source founder and he listed quite a few good points about how OSS companies of today are different from the previous decades:

Today companies are started with the OSS project and not many years after the project existed
They are created with an intention of sustainability and revenue generation. Developer users also want an OSS company to have revenue, unlike the fanatic developers of the past
The products are generally aimed at companies instead of individuals
Faster product improvement than in the past(because OSS projects used to be side projects). Also we aren't waiting around for contributors to volunteer, instead we are moving ahead to do the right things
Users have access to cloud hosting and APIs, so ease of getting started is 100x better than before
Real focus on design and UX. Well-designed OSS used to be an oxymoron, not today

I think that's very true :slightly_smiling_face:
Inspiration to use:
https://www.techrepublic.com/article/the-open-source-decade-fueled-by-cloud-and-github/
https://www.zdnet.com/article/open-source-turns-20/
https://www.consortiuminfo.org/open-source-open-standards/a-brief-history-of-open-source-software/

Inspiration to use:
https://www.techrepublic.com/article/the-open-source-decade-fueled-by-cloud-and-github/
https://www.zdnet.com/article/open-source-turns-20/

Draft: https://docs.google.com/document/d/13hM7ifUFFPG9jv7jOUFmbrHV1VB0JP-EATbmoRYtedY/edit

I don't see the link between the draft and the initial outline. I don't see any of the points made in the outline in the draft.
The point of this article that OSS companies have changed in the past 2-3 years from these 6 points mentioned.
Some examples of such new-generation OSS companies:

Airbyte
Posthog
Preset
dbt
Appsmith
Papercups
QuestDB

Here was the original outline: https://docs.google.com/document/d/1vCC-SPOgVMNfmff_egilV01-3te8MWi-gUwThIkvqVw/edit

@johnlafleur Totally agree! The article does not reference the OSS companies included in the outline. Instead, it references some general open source projects (not companies) like Linux, Firefox, Python, and GIMP.
They changed the title from "How Open-Source Companies Have Changed In The Last Decade" to "How Open Source Software Has Evolved" :/
I will check with Content Turbine to see what happened here...
I like your outline and see the value that could from there!


random

@PerteGeorge thank you for reporting this. I'll assign someone to take care of it soon.

@MihaiDejeu this is your task now, please go ahead. Deadline (when this ticket should be closed) is 2021-01-02T15:57:05.105428.
Estimation here is 60 minutes, that's how much you will be paid.
Remember, you don't have to solve everything in this ticket - you can solve it partially and leave todo markers in the code, which will become future tasks.
If you have any questions don't ask me, I'm not a technical person. Open new tickets instead.
If you don't have time or simply don't want to work on this, you can always resign.

@MihaiDejeu Don't forget to close this ticket before the deadline (2021-01-02T15:57:05). You are past the first half of the allowed period.

@MihaiDejeu thank you for resolving this ticket. I've just added it to your active invoice. You can always check all your invoices and more on the Contributor Dashboard.


Add kedro-airflow-k8s plugin to list of Community Developed Plugins

Description
We need to update our list of community developed plugins, as our ecosystem grows, this task focuses on adding the kedro-airflow-k8s plugin to our Community Developed Plugins page.
Possible Implementation
Follow the process in our Contribution Guide, and add a new bullet at the end of this list with the following:
- [kedro-airflow-k8s](https://github.com/getindata/kedro-airflow-k8s), [GetInData](https://github.com/getindata), a plugin that enable running a Kedro pipeline with Airflow on Kubernetes Cluster

Hey, can I get it?


Projections are ignoring the MapProperty attribute

Describe the bug
A clear and concise description of what the bug is.
A map property attribute is needed if properties have different names in a projection. When I add the attribute, it doesn't seem to do anything, while it does work if it's not a projection. If I add a regular mapping for the same types, then the regular mapping works AND the projection works!

To Reproduce
Steps to reproduce the behavior:

Create Dto and model classes with a property with mismatching names
Create a mapper with ONLY a IQueryable projection.
Add the Map property attribute to the projection.

public class ModelT
{
    public double Weight { get; set; }
}

public class ModelTDto
{
    public double Mass { get; set; }
}

[Mapper]
public partial class CarMapper
{
    [MapProperty(nameof(@ModelT.Weight), nameof(@ModelTDto.Mass))]
    public partial IQueryable<ModelTDto> Project(IQueryable<ModelT> query);

    //uncomment this code to make this work!
    //[MapProperty(nameof(@ModelT.Weight), nameof(@ModelTDto.Mass))]
    //public partial ModelTDto Dto(ModelT dto);
}

Result:
    public partial class CarMapper
    {
        public partial global::System.Linq.IQueryable<global::Moonshot.Agency.User.Mappers.ModelTDto> Project(global::System.Linq.IQueryable<global::Moonshot.Agency.User.Mappers.ModelT> query)
        {
#nullable disable
            return System.Linq.Queryable.Select(query, x => new global::Moonshot.Agency.User.Mappers.ModelTDto());
#nullable enable
        }
    }

Result when other mapper is uncommented:
    public partial class CarMapper
    {
        public partial global::System.Linq.IQueryable<global::Moonshot.Agency.User.Mappers.ModelTDto> Project(global::System.Linq.IQueryable<global::Moonshot.Agency.User.Mappers.ModelT> query)
        {
#nullable disable
            return System.Linq.Queryable.Select(query, x => new global::Moonshot.Agency.User.Mappers.ModelTDto()
            {
                Mass = x.Weight,
            });
#nullable enable
        }

        public partial global::Moonshot.Agency.User.Mappers.ModelTDto Dto(global::Moonshot.Agency.User.Mappers.ModelT dto)
        {
            var target = new global::Moonshot.Agency.User.Mappers.ModelTDto();
            target.Mass = dto.Weight;
            return target;
        }
    }

Expected behavior
A clear and concise description of what you expected to happen.
I would expect this to work even if the "Dto" method was not specified, like this:
    public partial class CarMapper
    {
        public partial global::System.Linq.IQueryable<global::Moonshot.Agency.User.Mappers.ModelTDto> Project(global::System.Linq.IQueryable<global::Moonshot.Agency.User.Mappers.ModelT> query)
        {
#nullable disable
            return System.Linq.Queryable.Select(query, x => new global::Moonshot.Agency.User.Mappers.ModelTDto()
            {
                Mass = x.Weight,
            });
#nullable enable
        }

    }

Code snippets
If applicable, add code snippets to help explain your problem.
Environment (please complete the following information):

Mapperly Version: 3.3.0-next.2
.NET Version: netstandard2.1
Target Framework: .net5.0
IDE: Version 17.7.4
OS: Windows 11 Enterprise  Version 10.0.22621 Build 22621

This is by design, as shown in the documentation: https://mapperly.riok.app/docs/configuration/queryable-projections/#property-configurations

Ah, gotcha. Do you know if there is a way to add logic to the projection? For example, if I had a car types in the database, something like below. I know you can add logic before and after the mapping, but that is more complicated with projections, as you can't modify with a select with essentially mapping twice. This is what I'm essentially trying to do (I can create another issue if that makes sense, but maybe I'm just missing something in the docs):
public class Car
{
    public double Weight { get; set; }
    public ModelT ModelT {get;set;}
    public ModelB ModelB {get;set;}
}

public class CarDto
{
    public double Mass { get; set; }
    public string CarType {get;set;}
}

[Mapper]
public partial class CarMapper
{
    public partial IQueryable<ModelTDto> Project(IQueryable<ModelT> query);

    [MapProperty(nameof(@Car.Weight), nameof(@CarDto.Mass))]
    [MapProperty("MethodNameToComputeCarType", nameof(@CarDto.CarType))]
    public partial ModelTDto Dto(ModelT dto);

  //essentially, trying to apply some logic to a specific property
   public bool MethodNameToComputeCarType(Car c) => c.ModelT != null ? "ModelT" : "ModelB";
}

@chrisevans9629 No, something like that is not yet supported. I currently also can't think of a workaround...
Please open a new issue so we can track this correctly :)

This is by design, as shown in the documentation

Is there a reason why or is this a design limitation?
I feel like ignoring members and MapProperty should be possible.

@TimothyMakkison If I remember correctly, the main reason was that it could get confusing if you had an IQueryable mapping plus a "normal" mapping method with the same types, but different configurations.
This probably needs to be documented better and may change in the future, if we find a good approach to this problem.

Could just be a warning where if you put a MapProperty on the IQueryable, it could say "This will be ignored. Please create a mapper for type A -> B with the desired attributes.

Created #868 to track this. Closing this, as the described behaviour is by design.


Clean Blog to Pelican version

I wonder if you could add the project link in the details of the layout. It is adapted to be used in the pelican
https://github.com/gilsondev/pelican-clean-blog

With the BS4 update I'll be re-reviewing all of the alternate versions of themes as they update to BS4.


set_snippet_length should apply when use electrode geometry is not checked

see https://www.figurl.org/fig?channel=franklab2&figureObject=b8272e7f30c2cf325ed9d3257cea10fac1e0998d&label=despereaux20191125_.nwb_02_r1_13 for an example where snippet length has been set to (20,20)

@lfrank I believe that the correct snippet len is actually being applied in both situations. Could it be an illusion of the display?

Ah, you’re right. Sorry for my error. Would it be possible to add a horizontal scale so that the waveforms could be compressed on the x-axis?

On Sep 27, 2021, at 7:51 AM, Jeremy Magland @.***> wrote:
@lfrank https://github.com/lfrank I believe that the correct snippet len is actually being applied in both situations. Could it be an illusion of the display?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/magland/sortingview/issues/148#issuecomment-927951105, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABV4PSKPFWQV3OM7WB4HPNLUECAH5ANCNFSM5EYI6XRQ.
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

I'll open an issue for that request.


The Remote Computer Refused The Connection: WinError1225

Installs went fine, disabled firewall and reset most network settings, still with the error. Just wondering if there is a fix, this was in reponse to the "python bootstrap.py login" command.

Did you use wand?

The command does not include that? What did you run exactly?

https://github.com/solero/wand

this usually means either you didn't install postgres or redis


[Power Behind the Throne] Corrupted image

The image for the Lore of Tzeentch journal looks to be corrupted.From<EMAIL_ADDRESS>5.0.5wfrp4e-archives1: 2.3.0wfrp4e-core: 2.3.3wfrp4e-dotr: 2.2.1wfrp4e-eis: 2.2.1wfrp4e-middenheim: 2.3.1wfrp4e-pbtt: 2.1.1wfrp4e-rnhd: 2.3.2wfrp4e-starter-set: 2.2.1wfrp4e-ua1: 2.2.1

Just as planned


SkillsDisplay: when autoScrollStrategy is configured to be top-offset last viewed scroll does not properly account for the offset

:+1:


Update cloudprovider terraform bazel file

What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?:

NONE

Thank you for the fix! I forgot to fix it🙏
/lgtm

Nice catch!
/approve


Calendar selection functionality is no longer work

Current Behavior
Calendar select functionality in both Project and Language Engagement is only displaying the current month and is not updating correctly for both start and end dates.
Expected Behavior
Should be able to scroll through years and/or select the desired month/day/year for both start and end dates of a project or a language engaement.
Steps to Reproduce

On local or Develop environment, navigate to a project
select the date range option for start/end date of a project
(also occurs inside a language engagement)

Screenshots

Environment
happens in local env and Develop

Closing this issue as it was resolved in https://github.com/SeedCompany/cord-field/commit/1c6ecdf6fa186a6bbf58177d71c2a0a6d2088698


Grpc query api

{grpcQueryAPI}

[ ] You've read the Contributor Guide and Code of Conduct.
[ ] You've included unit or integration tests for your change, where applicable.
[ ] You've included inline docs for your change, where applicable.
[ ] There's an open issue for the PR that you are making. If you'd like to propose a new feature or change, please open an issue to discuss the change or find an existing issue.

Summary of the changes
{Summary} (Less than 80 characters)
Description
{Detail}
Fixes #{issue number}

Closing due to no open related issue.


How to access char devices?

I a have a need to acce physical memory on arm device. I want to use memap to open /dev/mem but it fails with panic
thread 'main' panicked at 'failed to map the file: Error { repr: Custom(Custom { kind: InvalidInput, error: StringErro
r("memory map must have a non-zero length") }) }', /checkout/src/libcore/result.rs:906:4

It looks like /dev/mem has zero length for memmap. How can I handle that situation?

The man page for mmap says that mmap shall be supported for regular files, shared memory objects and typed memory objects. Support for any other type of file is unspecified.
Does /dev/mem fall into any of those three categories? It certainly doesn't seem to be a regular file.

Here is an example of usage mmap on /dev/mem: http://nairobi-embedded.org/mmap_n_devmem.html#system-ram-access
It seems to be very common approach to manupulate physical memory through the mmaping.

Don't you think that this fix is excess in unix.rs?
if aligned_len == 0 {
            // Normally the OS would catch this, but it segfaults under QEMU.
            return Err(io::Error::new(
                io::ErrorKind::InvalidInput,
                "memory map must have a non-zero length",
            ));
        }

Found solution.
For char devices programmer need to explicitly set length before mapping.
let mut mmap = unsafe {
	MmapOptions::new()
		.len(4096)
		.offset(0)
		.map_mut(&file).expect("failed to map the file")
};


TitleDescriptionModel description has both null and blank set

The TitleDescriptionModel is defined as:
class TitleDescriptionModel(models.Model):
    """
    TitleDescriptionModel

    An abstract base class model that provides title and description fields.
    """

    title = models.CharField(_('title'), max_length=255)
    description = models.TextField(_('description'), blank=True, null=True)

    class Meta:
        abstract = True

The description TextField has both blank and null set, this is an explicit anti-pattern in Django, see https://docs.djangoproject.com/en/3.0/ref/models/fields/#null

It's a shame that this would case a database migration for people.
However; since I am also in the process of removing Python 2 support and bumping the version to a next major version number (3.0.0) this might be a good time to change this as well.
If you could please make a PR with the change and add text to the documentation about this change and possible impacts like a database migration or people relying on the optional tri-state in the database.


CountDownTimer and SecondView Alpha Checker for better Performance added

Hello,
I've been using DraggablePanel with ExoPlayer but there were problems since I was using SENSOR when DraggablePanel was active, and I was using PORTRAIT on minimized. I needed to check SecondView Alpha and I was in need of Timers to make the isMaximized, isMinimized better. So I added the listeners below and improved when they are triggered:
onTouchListener
clickedToMaximized
clickedToMinimize
smoothSlide
The improvements also helped me to collect true data about how much DraggablePanel is used because it was throwing events way more since onMaximized and onMinimezed were called more than once in some cases.
You have to initialize timers when you are initializing the DraggableView with the code below:
draggable_view.initializeTimers();
My code for the Listener looks like below:
draggable_view.setDraggableListener(new DraggableListener() {
            @Override
            public void onMaximized() {
                if (!draggable_view.isClickedMinimize()) {
                    if (videoPlayer != null && prerollShown)
                        videoPlayer.showPrerollLayout();
                    if (draggable_view.isMinimized())
                        functions.sendEventAnalytics("panel", "maximized");
                    videoPlayer.setVisibility(View.VISIBLE);
                    setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_SENSOR);
                } else {
                    draggable_view.setClickedtoMinimize();
                }
            }

            @Override
            public void onMinimized() {
                if (!draggable_view.isClickedMaximize()) {
                    if (videoPlayer != null && prerollShown)
                        videoPlayer.hidePrerollLayout();
                    functions.sendEventAnalytics("panel", "minimized");
                    functions.colorComponentsNormal(MainActivity.this);
                    if (draggable_view.getSecondViewAlpha() < 0.15) {
                        new Handler().postDelayed(new Runnable() {
                            @Override
                            public void run() {
                                videoPlayer.setVisibility(View.VISIBLE);
                                setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_PORTRAIT);
                            }
                        }, 300);
                    }
                } else {
                    draggable_view.setClickedtoMaximize();
                }
            }

            @Override
            public void onClosedToLeft() {
                getWindow().clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
                functions.sendEventAnalytics("panel", "closed_to_left");
                draggableViewGone = true;
                if (videoPlayer != null)
                    if (videoPlayer.isPlaying())
                        videoPlayer.pause();
            }

            @Override
            public void onClosedToRight() {
                getWindow().clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
                functions.sendEventAnalytics("panel", "closed_to_right");
                draggableViewGone = true;
                if (videoPlayer != null)
                    if (videoPlayer.isPlaying())
                        videoPlayer.pause();
            }

            @Override
            public void onTouchListener() {
                draggable_view.setSeekBarStatus(videoPlayer.areContolsVisible());
                if (draggable_view.getSecondViewAlpha() < 0.99 && !draggable_view.isMinimized()) {
                    setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_PORTRAIT);
                } else if (draggable_view.isMinimized() && draggable_view.getSecondViewAlpha() > 0.1) {
                    setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_PORTRAIT);
                }
            }

            @Override
            public void clickedToMinimize() {
            }

            @Override
            public void smoothSlide() {
            }

            @Override
            public void clickedToMaximize() {
            }

        });

Best regards,
Renç Sarıdoğan.

Hi @rencsaridogan, thanks for your PR. I'm going to review it right now.
I've noticed that your PR is breaking the checkstyle rules I've configured. Can you review the code and format it to match with the checkstyle? If you want to do it automatically you can use this project to configure your IDE: https://github.com/square/java-code-styles

@pedrovgs I've implemented a change for anyone to be able to choose any rotation they like for the panel to work (Default is all), implemented the changes and applied the improvements we've discussed. ExoPlayer example is not added yet, working on that to prepare the best example to work on.
After the review you can merge this without waiting the example version. After the fixes and improvements it shouldn't effect the current version anyhow. Instead it should be adding the new features.
Best regards,
Renç Sarıdoğan.

Sorry for the delay @rencsaridogan, I've been out for some time. I think I can wait for the ExoPlayer example before to merge this PR. Please, if you have time add the example and the documentation needed to show how to use the new feature :)


Fix Go 1.15 regression in keystore tests

Go 1.15 has changed serialization of "SET OF" elements in ASN.1 to be sorted. This means that now the serialized form of exported key rings does not preserve the order in which they have been given to the Export() method. Therefore, we cannot depend on Import() to return their names after import in the same order. Replace the slice comparison with unordered comparison.
Some tests also depended on keys being exported and imported sequentially in the given order. This is no longer the case with Go 1.15 so tweak the tests to avoid such dependency.
This issue has been found by @iamnotacake in #410 and is actual only for Go 1.15+. This PR should wait until that one is merged, and bump the Go version to 1.15 for CircleCI to run the tests with up to date Go.

[ ] Upgrade Go to 1.15

Or you know... Maybe we can merge this PR first to support Go 1.15, and then #410 will bump Go to that version with linters.

That's a funny one. Thank you @iamnotacake for pointing out and @ilammy for fixing!


Possible to show the calendar inline without hiding the input?

This library is really great, but I'm wondering whether there's any combination of properties that might allow me to have the calendar picker open at all times while still showing the input? When I use inline: true, the input goes away. I've tried a few things but I feel like I'm missing something.
I'm working around it for now by mirroring the value of the hidden output, but it would be much nicer if I didn't have to do that.
Thanks!

Unfortunately not at this time


chore: set maven into batch mode for travisCI

prevent maven from download info flood

I didn't know you could do this - thanks!


how to associate HTML pages with CSS on github

I just started learning how to associate HTML pages with CSS on github.
Thank you for your time

Cool, let me know if you need any help.

Thank you, it has been solved. Because I just started to use github, the upload path of CSS was wrong, and it is normal after modification
I learned a lot through,developer.mozilla.org content is very good
Thank you again


Build McICA-sampled cloud mask from cloud fraction data

PULL REQUEST
Purpose and Content
Builds McICA-sampled cloud mask from cloud fraction data for maximum-random overlap
Benefits and Risks
(State concisely the benefits to be derived from and the risks associated with this PR)
Linked Issues
(Provide references to any link issues. Use closes #issuenum to automatically close an open issue)

Fixes #301
Closes #301

PR Checklist

[x] This PR has a corresponding issue OR is linked to an SDI.
[x] I have followed CliMA's codebase contribution and style guidelines OR N/A.
[x] I have followed CliMA's documentation policy.
[x] I have checked all issues and PRs and I certify that this PR does not duplicate an open PR.
[x] I linted my code on my local machine prior to submission OR N/A.
[x] Unit tests are included OR N/A.
[x] Code used in an integration test OR N/A.
[x] All tests ran successfully on my local machine OR N/A.
[x] All classes, modules, and function contain docstrings OR N/A.
[x] Documentation has been added/updated OR N/A.

bors try

bors try

bors try

bors r+


[BUG] spk-launcher only support unix

Describe the bug
spk-launcher only support unix and don't build under windows

use std::os::unix module to create symlink
use nix::unistd::execv to start a process

To Reproduce
Steps to reproduce the behavior:

run make build under windows
get error

error[E0433]: failed to resolve: could not find `unix` in `os`
  --> crates\spk-launcher\src/main.rs:8:14
   |
8  | use std::os::unix::fs::symlink;
   |              ^^^^ could not find `unix` in `os`
   |

and
error[E0432]: unresolved import `nix::unistd`
  --> crates\spk-launcher\src/main.rs:13:10
   |
13 | use nix::unistd::execv;
   |          ^^^^^^ could not find `unistd` in `nix`

Expected behavior
run make build under windows without error

This PR https://github.com/spkenv/spk/pull/1109 should fix it

The merged PR https://github.com/spkenv/spk/pull/1109 fix it


PATCH: Service generation Module fixes

Created and passed inference Test cases,
Test service factory fix and adding TASK class in the function of creating training rpcs,
Removal of usage of the function of get_module_info

After Pull request is made, I encountered a minor merge conflict which I resolved and committed.


Run reth in a private testnet

Describe the change
I want to run a private testnet using reth on two machines to debug the reth code I modified. But in the reth book, only one document uses kurtosis to build a private testnet. Can you provide a document for building reth from source code and building a private testnet with lighthouses?
Additional context
Run Reth in a private testnet using Kurtosis:    https://paradigmxyz.github.io/reth/run/private-testnet.html#run-reth-in-a-private-testnet-using-kurtosis

this should be doable with kurtosis if you use a locally build reth image instead of pulling it from ghrc

IMO the book does a good job at describing how to run a private testnet, just added a note in #8354


Resetting state

Hi,
I created multiple state components for some of the signingInStatus states, I have an issue when trying to go back from let's say, EXPIRED state to sign in view, since the sign in page has the conditional render for the expired state if I go back to it it just renders the expired link state component again because the signingInStatus is still on the EXPIRED state, I have to do a reload to reset state, is there any other way to achieve this?

Hi!
Can you share a bit more info on what exactly goes wrong?
Do you use the usePasswordless hook or also the prefab components?
Is the sign-in link still in the URL at that point? If you know you'll not need it (because it is expired) you may want to clear it.

For conditionally rendering the right UI use the signInStatus field btw not the signingInStatus field.
Just looking at the field signInStatus should be enough to figure out whether or not to show the right sign in page (was made for that purpose). Since being signed in (=have valid JWTs), and signing in, can exist in parallel (eg in case of step up auth), these statuses were separated into two different fields (they are related but not the same).

Hey @ottokruse , sorry for not getting back on this,

I'm using the usePasswordless hook

what I want is to show a different component for expired links vs link sent for example, now, when a link fails for whatever reason I also want to show a custom component with that feedback, and have a button to go back to sign in page/component but since the status is not updated without a reload, what it does is sends me back to failed link component since I'm rendering each of those components conditionally inside the sign in component based on the signingInStatus.

I'm using the signingInStatus instead of signInStatus because it offers me more specific statuses which is what I need for deciding which component to render.

the components would be something like this:
component A has => component B, C ,D, E
A
---- By default render B
---- if Expired render C
---- if Sent render D
---- if Invalid render E
----- if go back to sign in render A with cleared status so sign in form is shown

@ottokruse I ended up using a state to handle this, was hoping to be able to use the same hook but this way is not complex so I guess not really important to have such functionality.


Use liquid templating for stack ymls

---
stack: user/foo
variables:
  foo:
    type: string
    value: hello

services:
  bar:
    environment:
      # {% for num in (0...10) %}
      - FOO{{num}}={{foo}}/{{forloop.index}}
      # {% endfor %}

# $ irb -r ./lib/kontena_cli.rb
>> Kontena::Cli::Stacks::YAML::Reader.new('./foo.yml').execute[:services]["BAR"]["environment"]
=> ["FOO0=hello/1", "FOO1=hello/2", "FOO2=hello/3", "FOO3=hello/4", "FOO4=hello/5", "FOO5=hello/6", "FOO6=hello/7", "FOO7=hello/8", "FOO8=hello/9", "FOO9=hello/10", "FOO10=hello/11"]

Current implementation status:

Variables section is read independently, liquid sees ENV
Rest of the YAML can use the variables inside liquid

TODO:

Some magic for things like {{ service.mysql.hostnames | join ',' }}
What should happen if a service is scaled using kontena service scale?
The reader is messy, complete rewrite wouldn't hurt
It's a bit weird that you have a different set of variables in different sections of the YAML.
Unknown variables will output an empty string, even with the "strict_variables: true" setting in liquid.
There's no way to debug the interpolation/parsing output from the CLI.

Excellent! We need to add specs for this (and also test invalid liquid syntax).

https://github.com/Shopify/liquid/pull/832
Doesn't work with ruby 2.0 anymore.

Use cases
Just thinking what you would actually use it for.
Create conditional services
Already possible using the opto skip_if / only_if syntax
Iterate to generate multiple secrets
Opto could have something like:
variables:
  host_key:
    type: string
    instances: 5
    from:
      vault: ${STACK}-host-key-${INSTANCE_NUM}
      random_string: 128
    to:
      vault: ${STACK}-host-key-${INSTANCE_NUM}

services:
  foodb:
    instances: 5
    secrets:
      - secret: ${STACK}-host-key-%{INSTANCE_NUM} # server would have to eval % syntax
         name: HOST_KEY
         type: instance_env

Join instance hostnames
environment:
   UNICAST_HOSTS: {{ hostnames | join ',' }}

This could be done without liquid if such magical hostname variable (or opto resolver) was implemented.
This however would be difficult without liquid:
environment:
  # {% for instance in (1..foosvc_instances) %}
  - "UNICAST_HOST{{ instance }}=foosvc-{{instance}}"
  # {% endfor %}

If something requires such syntax, you could create a script in the container that would split a comma separated variable (or performs a dns lookup)
Iterate to generate hooks
This will (or already is) break service scaling unless some kind of instance hooks are created anyway. For something like building a fixed size mongo replica set this can work, but to me it doesn't seem like a very flexible practice.
Other
I haven't run into or can't think of any other use cases. Any ideas?

Not compatible with Ruby 2.0.0 is bad thing.. what can we do about it?

Needs rebase...

ping @jakolehm
cli/lib/kontena/cli/stacks/yaml/reader.rb cleaned up quite a bit.


Temporary build the leaderboard for December 2023

Handy to know how this works in case we need to repeat the exercise in the future - thanks for chasing this down Emil.

Closed as the results were published on January 8, 2023


Server stops accepting the connection

I have server on SSL, attached the certificate.
The server runs for almost 2-3 days and continuous clients are interacting suddenly the server stop accepting the connection.
Any idea what might be the reason?

Without more details, impossible to say. Got to try log an error or something.


how many training instances are used?

Hi, thanks for your great project!
I am wondering  how many training dataset instances you are used, such as COCO, OCR-VQA and A-OKVQA, did you just transform the original dataset with the template so the numbers are consistent with the original dataset?

I see the paper mention that 5k coco caption-image pair and 512 OCR, A-OKVQA pairs are used.
so if I am correct,y except for the LLAVA and minigpt4, there are 6k instances?


Server stopped after source code change in dev mode (Maven plugin)

Describe the bug
While using the liberty:dev Maven plugin, a source code change that triggered a restart lead to the server being stopped. Here are the last log lines:
[INFO] [2024-04-16T09:54:49.054+0000] 00000035 id=         com.ibm.ws.webcontainer.servlet.ServletWrapper               I doDestroy SRVE0253I: [app] [/] [...RestApplication]: Destroy successful.
[INFO] [2024-04-16T09:54:49.065+0000] 000003ad id=                                                                      R  Exception in thread "Thread-28"
[INFO] [2024-04-16T09:54:49.065+0000] 000003ad id=                                                                      R  java.util.concurrent.RejectedExecutionException: CWWKE1202E: A task cannot be submitted because the executor concurrencyPolicy[defaultConcurrencyPolicy] has been shut down.

[INFO] [2024-04-16T09:54:49.065+0000] 000003ad id=                                                                      R       at java.base/java.lang.Thread.run(Thread.java:840)
[INFO] [2024-04-16T09:54:49.089+0000] 00000035 id=         com.ibm.ws.app.manager.AppMessageHelper                      A CWWKZ0009I: The application app has stopped successfully.
[INFO] [2024-04-16T09:54:49.098+0000] 000003d9 id=         com.ibm.ws.webcontainer.servlet.ServletWrapper               I doDestroy SRVE0253I: [io.openliberty.microprofile.openapi.2.0.internal.servlet.jakarta] [/openapi] [ApplicationServlet]: Destroy successful.
[INFO] [2024-04-16T09:54:49.167+0000] 000003d9 id=         com.ibm.tx.jta.impl.FailureScopeController                   A WTRN0105I: The transaction service has shutdown successfully with no transactions requiring recovery.
[INFO] [2024-04-16T09:54:49.247+0000] 000003d9 id=         rty.security.mp.jwt.v21.config.impl.MpConfigProxyServiceImpl I CWWKS5788I: The MicroProfile JWT version 2.1 mpConfigProxy was deactivated successfully.
[INFO] [2024-04-16T09:54:49.249+0000] 000003d9 id=         com.ibm.ws.security.mp.jwt.impl.MicroProfileJwtConfigImpl    I CWWKS5502I: The MicroProfile JWT configuration [defaultMpJwt] was successfully deactivated.
[INFO] [2024-04-16T09:54:49.250+0000] 000003d9 id=         com.ibm.ws.security.mp.jwt.impl.MicroProfileJwtServiceImpl   I CWWKS5502I: The MicroProfile JWT configuration [MicroProfileJwtService] was successfully deactivated.
[INFO] [2024-04-16T09:54:49.391+0000] 000003d9 id=         ibm.ws.security.authentication.internal.jaas.JAASServiceImpl I CWWKS1124I: The collective authentication plugin with class name NullCollectiveAuthenticationPlugin has been deactivated.
[INFO] [2024-04-16T09:54:49.395+0000] 000003d9 id=         com.ibm.ws.security.ready.internal.SecurityReadyServiceImpl  I CWWKS0009I: The security service has stopped.
[INFO] [2024-04-16T09:54:49.710+0000] 00000001 id=         com.ibm.ws.kernel.launch.internal.FrameworkManager           A CWWKE0036I: The server defaultServer stopped after 4 minutes, 41.521 seconds.
[INFO] Server defaultServer stopped.

And the code involved:
@ApplicationScoped
public class Service {

    @Resource
    private ManagedExecutorService executor;

    public CompletableFuture<Void> runAsync(Runnable task) {
        return executor.runAsync(task);
    }
}

Expected behavior
The dev server should resist exceptions thrown during destruction.
Diagnostic information:

OpenLiberty Version: <IP_ADDRESS>
Java Version: 17

So @inad9300 it looks like the server is stopping because dev mode has exited.  I'm guessing it "crashed" because of some unexpected error/event that it couldn't handle.
Can you confirm that dev mode too exits when this issue happens?
Does this log message suggest anything to you?

[INFO] java.nio.file.NoSuchFileException: /app/src/test/java/.../TestUtils.java~

Assuming the understanding so far is correct, we could probably see what is going on more running with debug output (mvn -X) but that's likely not something you're going to want to run with most of the time waiting for this to happen.
Will wait for your response to consider what the next step might be.

Can you confirm that dev mode too exits when this issue happens?

Indeed, everything exits, and I have to rerun mvn liberty:dev.

Does this log message suggest anything to you?

It does, I should have mentioned – TestUtils has been one of my test files for a long time and never ceased to exist, but I was modifying it around the time the shutdown occurred. (At the same time, it's worth mentioning that it is the first time I see such a message, I never have before when the server crashes.)

TestUtils.java~ looks like a backup file that some editors create while editing a file.
For example, when I save a file named file on my machine with vim, it will rename the existing file to file~, write the updated contents to file and then delete file~.
I think the idea is to ensure that the old file contents are still available if the editor crashes while writing the new file contents but I wonder if having that short-lived TestUtils.java~ file is tripping up the file change detection in dev mode?

OK, we're getting somewhere.  Thanks for confirming.
Looking at the issue history, we actually added a filter for excluding file patterns from dev mode:
https://github.com/OpenLiberty/ci.common/pull/89/files.
(it's fixed in the common library used by both the Liberty Maven & Gradle plugins).
It'd seem we could add the ~ char at the end.
Unfortunately I don't think this is configurable, but rather hard-coded.
Let me look into what we could do to build you a test fix.  If you don't want to wait you could perhaps try building and testing a local snapshot as well (though it's complicated a bit by needing to build snapshots in each of the two repos).

Actually I guess it's more complicated than my last comment, since this code is already ignoring files that end with ~.

@inad9300 what version of the liberty-maven-plugin have you been using?  Apologies if you mentioned already but I'm not seeing it now.

I don't think I did, sorry. It's 3.10.2.

Hi @inad9300 still trying to think of how to make progress here, since I've been unable to recreate.
I'm wondering what OS you're running on?
As mentioned here, we have encountered problems on Linux with "devc" when using an editor (like 'vi') which does a delete and copy instead of updating the file in-place.
I can't think of why this would be a problem in regular 'dev' mode, but I wonder, if it would be convenient, if it would be worth running with set backupcopy=yes (if you are even using 'vi').  If you are encountering this frequently enough that you would probably notice a difference if this option were to improve things, that would be interesting.
Besides that another thing we might be able to do is to trigger some sort of thread dump from the NoSuchFileException.     I'm not sure how sensitive this application is, since that would likely effect how much detail you'd be willing to share in public.
(Or, if you have a support contract then you could share sensitive info through those channels).

Thanks for your persistence, despite how little there is to pull from. As an update, I experienced the problem again 2 or 3 times since my last message. Here are some more details about my setup, in case they help:

Operating system: Ubuntu 22.04.4 LTS
Editor: IntelliJ (Community)
I rarely see NoSuchFileException when this error occurs – in fact, I would say the only time was the one I shared last. Often I see no indication of exceptions at all, more like what I shared when I opened the issue.

Thanks for the additional info.
So if we were able to come up with a strategy using dump agents to collect say something like the Open J9 javadump, would you be willing/able to share that publicly?  (Or would you be able to open a support case with IBM and share privately?)

Sorry for dodging your question earlier. Hmm... :thinking:  No, I don't think so. We'll have to come up with an alternative way to debug this, sorry.

OK, a couple more questions:

Have you detected any kind of pattern where editing certain kinds of files led to this error?  E.g. was it always TestUtils?  Always test classes?  Always Java parts, etc?
Can you describe your project setup a bit?   Is it single-module or multi-module?  If the latter, what module does this TestUtils class live within (is it the module with the Liberty server config or something else, like a "utility" JAR module, say)?

Thanks

No, I have not. It happens in different contexts every time, I would say.
It is a single module, where TestUtils lives. We do have a Git submodule which happens to be another Java project, though this is likely not relevant.

Hmm... you might try to run with mvn -X liberty:dev  and at least see if it slows you down noticeably.   If it isn't too slow, maybe you could try running with it regularly until you recreate the issue?      At that point, even if you aren't willing to share the entire log maybe you could share enough of an excerpt to suggest the next step here.
If you do decide to try this, please remember to redirect the log to a different directory outside the current project, or else dev mode will watch and react to the log changes!    E.g. something like mvn -X liberty:dev | tee ../mvn.log

I have been trying to use -X (formerly -Twitter 🙈) a lot for the past months,

Thanks for the update.   One doesn't see a lot of humor in Maven plugin issue comments, so I tip my hat 👏 for that 😃 .
Yes, that's definitely a blunt instrument so not surprised it changes the timing and is hard to tolerate too often.
Will keep this open and see if we can think of anything.


How to use nearest neighbor interpolation?

When sampling from a 64 x 64 texture (Bayer matrix), I get the below in the preview.

However, I expected a matrix-like texture. Any idea how to switch to nearest neighbor?

@VanushVaswani  glsl-canvas supports { filtering: 'nearest' } as texture option, but due to the way vscode-glsl-canvas plugin currently handle textures is not possible to pass parameters. I'm trying to add the feature with those two proposal:
proposal 1
adding texture options as evaluable object after texture url
uniform sampler2D u_texture; // data/bayer-matrix.png : { filtering: 'nearest' }

proposal 2
adding texture options as querystring of the texture url
uniform sampler2D u_texture; // data/bayer-matrix.png?filtering=nearest

latter proposal could work with vscode glsl-canvas settings too:
{
    "glsl-canvas.textures": {
        "0": "./data/bayer-matrix.png?filtering=nearest",
    }
}

I like proposal 2!

@VanushVaswani proposal 2 has been added to latest plugin release! Thank You!

Is there a way to set wrap/repeat modes with this method as well?


Fixes GH-351

I feel uneasy about deleting those kind of lines.
Logically, it doesn't make sense why it's there...I'm trying to think of a potential use case where applying this fix will break JSHint but I can't think of any.
The nospace is already handled prior to checking if the nexttoken is ) so I'm not sure why we need to call nospace again. Just advancing the token and returning should be fine.

GH-351

Thanks! Merged in.


Narrow the scope of "multicheck" modification checker

See PR https://github.com/aristath/kirki/pull/562

Thanks! merged :+1:


Fix for NSD Attachment blocking

Workaround for NSD Services being a disaster and resulting in many-second blocking calls. Now will abandon the search after a brief window (it won't succeed after it's gotten into the bugged state anyway)
Ticket:
http://manage.dimagi.com/default.asp?222766#1120157

@phillipm Feedback addressed, thanks for review!


Add FoundIT Review post

I'll be writing about my thoughts on building FoundIT and what I've learned

Closed in https://github.com/projectunic0rn/pub/pull/56


CRAYSAT-1750: Allow PyYAML >= 6.0

Summary and Scope

CRAYSAT-1750: Allow PyYAML >= 6.0
Update version constraints in pyproject.toml to allow PyYAML >= 6.0.
This allows packages which require PyYAML 6.0.1 to address the Cython
3.0 incompatibility to continue building.
The main changes between PyYAML 5.x and 6.x are:

yaml.load always requires a Loader argument
Python 2.7 is no longer supported
YAMLLoadWarning is gone

Only this last one affects this code. This is fixed by dropping the code
that catches the YAMLLoadWarning. We no longer need to catch this
warning anymore. The kubernetes library has not emitted that warning
since v10.0.0, and it certainly won't if installed with PyYAML 6.0.1.
Test Description:
Unit tests pass. Will be tested in sat before inclusion in a release.

CRAYSAT-1750: Preserve Python 3.8 compatibility of type hints
Python 3.8 does not allow subscripting of builtins.list and the special
class typing.List must be used instead. Python 3.9 adds support for
subscripting list and deprecates typing.List.
For consistency with existing code and to preserve compatibility with
the older Python version, just use typing.List for type hints.
Test Description:
Unit tests fail on Python 3.8 without this change and succeed with this
change.

CRAYSAT-1750: Update to newer poetry version
Update the build pipeline to use a newer version of poetry. This should
be a non-disruptive change, and it's good to stay up to date. This is
the latest version which was installed on my macbook when I installed
following the official install instructions.
Also update the pyproject.toml file to use the new way of specifying
additional package sources. See documentation of repositories here:
https://python-poetry.org/docs/repositories/#supplemental-package-sources
Test Description:
Installed the package with poetry install locally on my macbook.

Issues and Related PRs

Resolves CRAYSAT-1750

Testing
See descriptions above.
Risks and Mitigations
These library is only used by sat right now, and it's a pretty small change to requirements, so it's pretty low risk.
Pull Request Checklist

[x] Version number(s) incremented, if applicable
[x] Copyrights updated
[x] License file intact
[x] Target branch correct
[x] CHANGELOG.md updated
[x] Testing is appropriate and complete, if applicable
[x] HPC Product Announcement prepared, if applicable

My recent force push was just fixing my previous attempt to declare a version. I had forgotten to update the version in pyproject.toml while updating the version in CHANGELOG.md. I also split the changelog and version changing into its own commit since it really applies to all three other commits in this PR.
I also wanted to note that one side effect of updating the version of Poetry used in the build is that the python package is now published as csm_api_client-1.1.5.tar.gz instead of csm-api-client.tar.gz as it would have been with the older version of Poetry. There is some discussion of this change in the following Poetry issue: https://github.com/python-poetry/poetry/issues/6915 (backticks to prevent unnecessary linking).

I have tested this in a build of the cray-sat container image. I ran the following commands with the cray-sat container image:

sat showrev --products
sat bootprep run bootprep.yaml where the bootprep.yaml file defined a single configuration with a single product-based layer. This excercised YAML loading, Kubernetes access within the python-csm-api-client library, and access to the CFS API to create a configuration.
sat status. This exercised access to multiple APIs using the python-csm-api-client.

/backport release/1.1


Changed this._nodes to an array and added an addElement() method.

The hope is that addElement() can handle use cases where img elements are created well after Layzr is initialized. One could call updateSelector() in this scenario, but if you already have access to the element it might be simpler to add it to the nodes Layzr references.

This jsfiddle is probably way clearer than what I typed above :) http://jsfiddle.net/maxnachlinger/zqj7vy0j/

@maxnachlinger hey man. thanks for submitting this, and throwing together the JS fiddle. couple of my thoughts:

I don't see how it's easier than calling updateSelector - its usefulness is limited, because it requires having the image node handy
the performance benefit adding singular nodes is outweighed by the overhead of always needing to convert the nodelist updateSelector returns to an array

for those reasons I think I'll take a pass, but again, really appreciate the interest/engagement

Np, thanks for having a look.


created a todoist plugin

I've created a simple todoist plugin as it's another tool I'd like to be combined within franz.

I'm also interested in this


Sensible default for RoleAppointment and Government start dates

The common use-case is adding a new role appointment at the time
they appointment is made, and it makes more sense to set the
default started_at date to be today. For historic role appointments,
the default is left blank.

When creating a new government the common case is a modern
government that has just come to power. Set the default
value for government start_date to be today, so someone
adding a new government in the common case has less
work to do.

https://trello.com/c/7IVzcDP6/147-when-creating-new-roles-and-governments-set-the-default-date-to-today
No screenshots as they'd be pretty dull.

LGTM

Great stuff David.


NOISSUE - Fix bug on same dataset provider for multiple datasets

What type of PR is this?
This is a bug fix of dataset provider upload order

What does this do?

New Features

Improved user authentication process for better context handling.
Enhanced dataset management with error handling for undeclared datasets.

Bug Fixes

Resolved issues related to dataset validation and filename consistency.

Refactor

Simplified context handling in user authentication for clarity and efficiency.

sequenceDiagram
    participant User
    participant AuthService
    participant DatasetService

    User->>AuthService: AuthenticateUser(ctx, role)
    AuthService-->>User: Return original ctx

    User->>DatasetService: Data(ctx)
    DatasetService->>DatasetService: Iterate through datasets
    DatasetService->>DatasetService: Check hash and filename
    alt Match found
        DatasetService-->>User: Dataset retrieved
    else No match
        DatasetService-->>User: ErrUndeclaredDataset
    end

Which issue(s) does this PR fix/relate to?

Have you included tests for your changes?
manually tested

Did you document any new/modified feature?
no

Notes

➜  cocos git:(single-data-provider) ✗ export AGENT_GRPC_URL=localhost:6019
➜  cocos git:(single-data-provider) ✗ ./build/cocos-cli algo ../ai/covid19/train.py private.pem -a Python -r ../ai/covid19/requirements.txt
2024/08/16 14:07:12 Uploading algorithm binary: ../ai/covid19/train.py
Uploading algorithm...  100% [===============================================>]
2024/08/16 14:07:12 Successfully uploaded algorithm
➜  cocos git:(single-data-provider) ✗ ./build/cocos-cli data ../ai/covid19/datasets/h3.zip private.pem

2024/08/16 14:07:21 Uploading dataset CSV: ../ai/covid19/datasets/h3.zip
Uploading data...  100% [====================================================>]
{"time":"2024-08-16T14:07:22.877016164+03:00","level":"ERROR","msg":"Failed to send Data"}
2024/08/16 14:07:22 Error uploading dataset: rpc error: code = Unknown desc = agent not expecting this operation in the current state

➜ cocos git:(single-data-provider) ✗
➜  cocos git:(single-data-provider) ✗ export AGENT_GRPC_URL=localhost:6005
➜  cocos git:(single-data-provider) ✗ ./build/cocos-cli algo ../ai/covid19/train.py private.pem
2024/08/16 14:10:23 Uploading algorithm binary: ../ai/covid19/train.py
Uploading algorithm...  100% [===============================================>]
2024/08/16 14:10:23 Successfully uploaded algorithm
➜  cocos git:(single-data-provider) ✗ ./build/cocos-cli data ../ai/covid19/datasets/h3.zip private.pem

2024/08/16 14:10:27 Uploading dataset CSV: ../ai/covid19/datasets/h3.zip
Uploading data...  100% [====================================================>]
2024/08/16 14:10:29 Successfully uploaded dataset
➜  cocos git:(single-data-provider) ✗

Does not work when algo type is specified

share your steps here and manifest as well, as I am not able to replicate this behavior


How to downgrade? + How to turn off upgrades?

Is there a way to downgrade from 25 to 24 and make it stay (so it doesn't upgrade by itself)?
frzr-deploy seems to only understand stable/unstable and not a specific version, so how else can I do it, maybe manually (download gameros-24_d5ee4b6.img.tar.xz but then what)?

Downgrades are not supported. But what specific bug do you have that requires a downgrade?

I'm just tired of fixing stuff after every upgrade (thus the question how to turn them off). This time specifically, all proton games and flatpack games (minecraft...) stopped working. I tried some of that: https://github.com/ChimeraOS/chimeraos/issues/247 but with not luck. I figured I could just downgrade to the last-working-version but apparently not...

When Nvidia drivers are upgraded, you have to re-install Minecraft so that flatpak picks up the latest Nvidia driver updates.
For proton games, you may be affected by an issue in Steam. Try disabling shader caching.

Can you test if in current testing build the problem persists? (You can go back from testing to stable)
$sudo frzr-deploy chimeraos/chimeraos:testing
to go back: $sudo frzr-deploy chimeraos/chimeraos:stable

"When Nvidia drivers are upgraded, you have to re-install Minecraft so that flatpak picks up the latest Nvidia driver updates."
that was the case with previous upgrades and of course I did that, but didn't help this time
"Can you test if in current testing build the problem persists? (You can go back from testing to stable)"
yes, I did that before creating the issue, no, it didn't change anything :-(
All proton games and minecraft were stuck on the loading screen (the blue one with three blinking stripes). Native games work normally.
Now, I reverted all the changes and redeployed the stable version, as well as opted-out of steam beta, minecraft works but proton games give the

So anyway, getting back on track, what is the best way to turn off system upgrades? Can I just remove/rename some script etc?

To stop updates entirely you need to disable the frzr-autoupdate.timer from systemd.
I would highly recommend against doing that. The best course of action is to figure out what are your issues and work them out for future releases so they won't affect you (and potentially other users).


Initial coarse cell support.

Tested with 200k+ design. 60k @or cells converted to 6k @reduce_or cells. Might be useful if you can ask from gate.is_single_bit().

@rqou Can you review this PR please?

Latest version doesn't assert and seems to find reduce_* cells from big design, but from mini example I posted here it doesn't find.
Please could you @rqou help with this.

+1
Looks like it works, and even fixed a bug that I had.

Fixed as requested.

Thanks. The PR currently has merge conflicts with Yosys master. Can you pull (merge) master and resolve the conflicts? Thanks.

Wondering... did I make this pull(merge), resolve as it supposed to be. Learning to use git...

Current head version of extract_reduce is quite different so it's easier to make new PR based on it.


[Fix] Force the inputs of get_bboxes in yolox_head to float32.

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.
Motivation
Force the inputs of get_bboxes in yolox_head to float32.
Sovle #6976
Modification
Please briefly describe what modification is made in this PR.
BC-breaking (Optional)
Does the modification introduce changes that break the backward-compatibility of the downstream repos?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.
Use cases (Optional)
If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.
Checklist

Pre-commit or other linting tools are used to fix the potential lint issues.
The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMCls.
The documentation has been modified accordingly, like docstring or example tutorials.

Kindly ping @chhluo and @hhaAndroid to have a look


[Merge after #13] Add decode function.

What does this pull request do?
The purpose of this pull request is to extend the library with a function that decodes form data into the FormURLEncoded structure.
Where should the reviewer start?
It's a pretty small change.
How should this be manually tested?
Here are my test cases:
> encode $ decode "a=aa&b=bb"
"a=aa&b=bb"

> encode $ decode "this=this%3Dthis"
"this=this%3Dthis"

> encode $ decode "&x=x&&y=y&z="
"&x=x&&y=y&z="

> decode "a=aa&b=bb"
(FormURLEncoded [(Tuple "a" (Just "aa")),(Tuple "b" (Just "bb"))])

> decode "this=this%3Dthis"
(FormURLEncoded [(Tuple "this" (Just "this=this"))])

> decode "&x=x&&y=y&z="
(FormURLEncoded [(Tuple "" Nothing),(Tuple "x" (Just "x")),(Tuple "" Nothing),(Tuple "y" (Just "y")),(Tuple "z" (Just ""))])

Other Notes:
I already updated the README, and I don't think any follow-up steps are necessary other than cutting a release.

Thanks for the response @thomashoneyman. I should have given more consideration to the possibility of decodeURIComponent throwing an error. It turns out that both encodeURIComponent and decodeURIComponent can throw errors although purescript-form-urlencoded's encode function doesn't currently have any notion of failure either. I also looked at purescript-uri's usage of unsafeDecodeURIComponent and usage of unsafeEncodeURIComponent and found no error handling there, either, so I am not entirely sure how to proceed.
I think the main issue is that the type of unsafeDecodeURIComponent and unsafeEncodeURIComponent, i.e. String -> String, doesn't suggest possible failure. So we are left to find a safe alternative, perhaps, to purescript-globals?
Another option would be to resort to the FFI as in purescript-httpure, but, actually, is this modeled correctly? I thought that throwing an error is considered a side effect. (As a relative PureScript newbie, I would love clarification one way or the other!) Assuming that throwing an error is a side effect, I think this would ultimately need to be reflected in the return types of encode and decode, or else we would need to resort to unsafePerformEffect perhaps...
Yet another possibility would be to attempt to validate the input before calling unsafeEncodeURIComponent/unsafeDecodeURIComponent, hopefully avoiding the possibility of a URIError altogether. I have found some examples attempting to do this sort of thing (here, here, and here), but I do wonder about the correctness/exhaustiveness of these solutions.
Please let me know what you think, or if you had another solution in mind. Maybe @garyb would be willing to opine as well, since I believe he has worked on purescript-uri and can see that he is keeping an eye on this issue. Otherwise, thanks again for the feedback!

purescript-uri makes use of the unsafe functions "safely" :wink: - it only uses them in a context where the string has been parsed already - they're basically used just as lookup tables to do the actual decode conversion for characters, when we know they're already using a valid encoding. Even the encode stuff isn't used naively - none of the browser-provided encode functions entirely do the right thing for all parts of a URI, so they're only used to encode some characters - again just being used as a lookup really.
I agree that ideally there should be non-unsafe version of en/decodeURI[Component], I think the reason they don't exist in -globals is it's a zero-dependency library. If I had Maybe on hand, I'd implement them to return a value that models the exception that way - there's not any interesting information in the error or stack trace, so just a straight yay-or-nay result would work I think. Maybe it's time -globals got a dependency, I don't know.
And yeah, an exception/Error is most definitely a side effect, and should correspondingly happen in Effect. 🙂

Thanks for the comprehensive response! @garyb made some good points; here are a few thoughts on my end:

It turns out that both encodeURIComponent and decodeURIComponent can throw errors although purescript-form-urlencoded's encode function doesn't currently have any notion of failure either.

This sounds like a problem to me. If encoding and decoding can fail then their types should reflect that -- likely through a Maybe return type.

I also looked at purescript-uri's usage of unsafeDecodeURIComponent and usage of unsafeEncodeURIComponent and found no error handling there.

@garyb noted that there isn't error handling because the author(s) felt confident that they were providing input which would always succeed. So that's a sensible use of the unsafe functions. But it doesn't mean other code should do the same thing.

I think the main issue is that the type of unsafeDecodeURIComponent and unsafeEncodeURIComponent, i.e. String -> String, doesn't suggest possible failure. So we are left to find a safe alternative, perhaps, to purescript-globals?

The use of unsafe* generally denotes "I might throw errors or otherwise do things not ordinarily allowed in PureScript given my type, so be careful," so I don't mind this pair of functions existing with their current types (and disclaimers that they can throw errors).
I think you've raised a good point -- if these are defined in the purescript-globals package, why not update that package's main namespace (Global) to have safe versions of the unsafe module (Globals.Unsafe)? That way we can have decodeURI :: String -> Maybe String and not have to worry about it.
@garyb I don't see a reason why -globals should avoid a dependency on -maybe; it's used basically everywhere else. Is there a reason it ought to be dependency-free?

Another option would be to resort to the FFI as in purescript-httpure, but, actually, is this modeled correctly? I thought that throwing an error is considered a side effect.

Generally that's true -- throwing an error is a side effect -- but a try/catch used in an FFI file is more akin to Maybe. Using it the way HTTPure does is like the function returning null on failure. That makes me comfortable representing the function as String -> Maybe String in PureScript. Perhaps it's not technically correct, but I'd still consider that FFI implementation a non-effectful function.

My vote would be:

Make safe versions of encodeURIComponent and decodeURIComponent in purescript-globals
Implement them the way HTTPure does, with the try/catch FFI implementation
Import and use them in this library

Updated test cases:
> decode "a=aa&b=bb"
(Just (FormURLEncoded [(Tuple "a" (Just "aa")),(Tuple "b" (Just "bb"))]))

> decode "this=this%3Dthis"
(Just (FormURLEncoded [(Tuple "this" (Just "this=this"))]))

> decode "&x=x&&y=y&z="
(Just (FormURLEncoded [(Tuple "" Nothing),(Tuple "x" (Just "x")),(Tuple "" Nothing),(Tuple "y" (Just "y")),(Tuple "z" (Just ""))]))

> decode "a=b&%8A=c"
Nothing

> encode =<< decode "a=aa&b=bb"
(Just "a=aa&b=bb")

> encode =<< decode "this=this%3Dthis"
(Just "this=this%3Dthis")

> encode =<< decode "&x=x&&y=y&z="
(Just "&x=x&&y=y&z=")

> encode =<< decode "a=b&%8A=c"
Nothing

@garyb this is going to be a breaking change because of the type signature difference. Do you have an opinion on adding unsafeEncode and unsafeDecode functions for folks relying on this to use in the next version? Or should they just use unsafePartial to recover that?

Updated test cases:

@nsaunders actually, do you mind adding a test directory with those tests you've written? It would look something like:
https://github.com/purescript-contrib/purescript-js-date/blob/master/test/Test/Main.purs
https://github.com/thomashoneyman/purescript-slug/blob/master/test/Main.purs
where each of your lines above would be an assert statement. I can then set up CI which will run the tests automatically for future branches.

@thomashoneyman yeah, I'd release it with a new breaking version. The old one is "bad" since it can explode, so it'd be better if people updated to the new one and dealt with the fact that encoding can fail 🙂
👍 for adding some tests that run in CI

I've added an issue, #14, for the test cases. I've also updated master to include a test directory and for CI to run those tests.


Feature Request: Automated and controllable version increments for semver

Is your feature request related to a problem?
I'm unsure whether this feature request is for documentation for an example (in case I'm not using the settings correctly) or indeed a new feature.
Problem statement: to reduce risk and churn, we want to assess "major" upgrades on a different cadence from "patch" upgrades.
For example, if upgrading from version 4.x.x to 5.x.x, we want to review on a monthly cycle so that we can ensure there are no breaking changes and test accordingly. Additionally, some services, like Istio or Kubernetes, have significant changes in "minor" releases (e.g. 1.17.0 to 1.18.0) so we would like to update the minor releases monthly, and update patch releases otherwise.
For minor releases (e.g. Istio 1.17.1 to 1.17.2), we would like to review and apply patches on a weekly basis, since the risk should be low and we would pick up security patches and bug fixes as part of this.
Solution you'd like
Ideally (and perhaps versionIncrement already does this), we would have a way to specify that we are willing to accept minor updates or patch updates for a given source. Then, if the current version is 1.17.1, and upstream available versions are 1.17.2 and 1.18.0, updatecli should choose 1.17.2.
Alternatively, some way to support inheritance between two updatecli configs (similar to Docker Compose's extends syntax) would be useful. In this case, we could reduce duplication by having our minor update config extend our major update config.
Alternatives you've considered
We are currently using two separate configuration files.
In the config file that we run monthly, we use:
sources:
  cert_manager:
    kind: helmchart
    spec:
      url: https://charts.jetstack.io
      name: cert-manager
      versionFilter:
        kind: semver

targets:
  cert_manager:
    name: bump chart dependencies
    kind: yaml
    spec:
      file: apps/cert-manager/helmrelease/cert-manager.yaml
      key: $.spec.chart.spec.version
    sourceid: cert_manager
    transformers:
      - trimprefix: v

Note that we are not using kind: helmchart for the target because we are updating a file containing a HelmRelease (a Custom Resource Definition provided by Flux CD)
In the config file that we run weekly, we instead add a version filtering pattern:
sources:
  cert_manager:
    kind: helmchart
    spec:
      url: https://charts.jetstack.io
      name: cert-manager
      versionFilter:
        kind: semver
        pattern: 1.*.* # this is the only difference

targets:
  cert_manager:
    name: bump chart dependencies
    kind: yaml
    spec:
      file: apps/cert-manager/helmrelease/cert-manager.yaml
      key: $.spec.chart.spec.version
    sourceid: cert_manager
    transformers:
      - trimprefix: v

This works well, but requires us to keep these two files in sync and there is significant duplication otherwise.
Anything else?

I've tried using versionIncrement: minor,patch in both source and target blocks, but have not been able to achieve the desired behavior. This may be a misunderstanding on my part about what this feature does and how it works.
I've tried examining the source code but was unable to determine whether there was a way to get updatecli to exhibit the desired behavior: https://github.com/updatecli/updatecli/blob/4fa6b0e9dc8c8a1f65dc9ac3161692640567ceb5/pkg/plugins/utils/version/filter.go#L114-L125
I've tried using the semverinc transformer on the helmchart source, but also might be misunderstanding how this works: https://github.com/updatecli/updatecli/blob/4fa6b0e9dc8c8a1f65dc9ac3161692640567ceb5/pkg/core/transformer/main.go#L148-L163

There are multiple things, I'll try to treat them separately to give you actionnables (I believe @olblak might also have ideas and feedbacks eventually).

First of all, you should change versionFilter to versionfilter. The documentation / example might not up to date about this.
I recommend you to use the JSON schema validation when editing your manifests (it is out of the box in VSCode) to point such syntax changes to the end users.
The output of updatecli most probably warns you when executing a diff or an apply.
I'm pointing this out as some features might be deprecated and disabled with the older syntax.

About the pattern for version filters of type semver are documented here: https://www.updatecli.io/docs/plugins/_versionfilter/ but lacks examples and/or link to the library we use under the hood: https://github.com/Masterminds/semver.
This library does not allow a direct syntax to "only" request a patch or minor without writing a specialized pattern (https://github.com/Masterminds/semver#tilde-range-comparisons-patch). But you can express some elements such as:
# Only update to latest patch of the 1.17 line (don't update to 1.18.x)
# ref. https://github.com/Masterminds/semver#tilde-range-comparisons-patch
versionFilter:
  kind: semver
  pattern: '~1.17'

# Only update to latest patch of the 1.x line (don't update to 2.0.0)
# ref. https://github.com/Masterminds/semver#caret-range-comparisons-major
versionFilter:
  kind: semver
  pattern: '^1.x'

But you must have some prior knowledge to update (in these case you must keep the values 1.17 and 1 updated). Note that you can use the second manifest (the one trackin the major version monthly) to update the first ;)

Then, about the versionIncrement:
The documentation at https://www.updatecli.io/docs/plugins/autodiscovery/helm/ (check the list of parameters, last item of the array) mentions that it is only when using a target of kind helm. It's used to auto-increment a chart version when the chart is updated by updatecli.
The documentation is being updated since a few weeks so this might not have been easy to catch, no worries. But I'm pointing it to you to avoid having you wasting your time on this attribute for your use case, since your targets are CRDs (e.g. yaml file).

About the "code duplication":
There are a lot of discussion and back and forth work about this in updatecli and your testimony here really helps to provide real life use cases thanks!
I remember ideas (long ago) of @olblak about multi-valued sources (but only sources).
There are multiple tools to help you already:

YAML anchors if you choose to have a single manifest

Involves expressing 2 sources (one for patch and one for major for instance) and 2 actions of type pull requests (one for each). But with an anchor, that could help you to reduce the duplication:

---
name: Bump Cert-Manager Version

scms:
  cert_manager_patch: &scmSpec
    kind: github
    spec:
      user: "{{ .github.user }}"
      email: "{{ .github.email }}"
      owner: "{{ .github.owner }}"
      repository: "{{ .github.repository }}"
      token: "{{ requiredEnv .github.token }}"
      username: "{{ .github.username }}"
      branch: "{{ .github.branch }}"
  # Exactly the same
  cert_manager_minor: *scmSpec

sources:
  cert_manager_patch: &sourcePatch
    kind: helmchart
    spec: &patch
      url: https://charts.jetstack.io
      name: cert-manager
      versionfilter:
        kind: semver
        pattern: '~1.12'
  cert_manager_minor:
    kind: helmchart
    spec:
      <<: *patch
      versionfilter:
        pattern: '^1.x'

targets:
  cert_manager_patch: &targetPatch
    name: Bump Cert Manager Patch Version
    kind: yaml
    sourceid: cert_manager_patch
    spec:
      file: .github/workflows/updatecli.yaml
      key: $.name
    transformers:
      - trimprefix: v
    scmid: cert_manager_patch
  cert_manager_minor:
    <<: *targetPatch
    name: Bump Cert Manager Minor Version
    sourceid: cert_manager_minor
    scmid: cert_manager_minor

actions:
  cert_manager_patch:
    kind: github/pullrequest
    scmid: cert_manager_patch
    spec:
      title: Bump Cert Manager Patch Version to {{ source `cert_manager_patch` }}
      labels:
        - cert-manager
        - patch
  cert_manager_minor:
    kind: github/pullrequest
    scmid: cert_manager_minor
    spec:
      title: Bump Cert Manager Patch Version to {{ source `cert_manager_minor` }}
      labels:
        - cert-manager
        - minor

I believe we could discuss this feature in the long term under the following scope:

Only for source,
a collection of versionFilter could be defined
Each one of these filters could be referenced later

Something like this:
sources:
  cert_manager:
    kind: helmchart
    spec:
      url: https://charts.jetstack.io
      name: cert-manager
      versionfilters:
        patchFilter: # User-specified name
          kind: semver
          pattern: '~1.12'
        cert_manager_minor:
          kind: semver
          pattern: '^1.x'
    transformers:
      - trimprefix: v

targets:
  cert_manager_patch:
    name: Bump Cert Manager Patch Version
    kind: yaml
    sourceid: cert_manager.patchFilter
    spec:
      file: .github/workflows/updatecli.yaml
      key: $.name

WDYT @olblak ?

Side-tracking: Specifying 2 different PRs from 2 different targets looked really cumbersome in my previous message @olblak: do you think we could have a way to link actions and targets other than using the scmid? I have vague memory of playing with this in the past but can't remember if it is possible today

That's a lot of information to process, so I'll try answer them one by one
Version Increment
Writing a policy that would do what you ask has been puzzling me for a while :) .
To know the "latest" version, we need to know the current version, the update pattern from that version, to finally retrieve the "latest" upstream version
Transformers
One of my first attempt to solve this was by introducing transformers of type semverinc such as in the following example
Example
sources:
  default:
    kind: yaml
    spec:
      file: /tmp/data.yaml
      key: $.version

conditions:
  default:
    kind: dockerimage
    spec:
      image: ghcr.io/updatecli/updatecli
    transformers:
      - semverinc: minor
      ## semver drops v prefix
      - addprefix: v

targets:
  default:
    kind: yaml
    spec:
      file: /tmp/data.yaml
      key: $.version

Two major drawbacks:

We probably want to update to the latest version and not one version at a time.
The pipeline fails if there is no new version, which is quite confusing :(.

But semverinc ended up being useful in some situation so I kept it and regularly use it.
Autodiscovery
In the autodiscovery context, it was easier because we analyze the files we want to update and then all we need to do is to generate the right semver pattern.
An example here
Once again they are drawbacks
.1 We could have two similar pullrequests opened at a time, one bumping the patch version and another one bumping the minor version.
-> It's mitigated in your case because the minor bump happened once a month while the patch bump happened every day
-> In my scenarii, I automerge patch update or dependencies that I trust and specifically don't automerge those that I don't trust.
.2 More autodiscovery plugins means more code to maintain. :(
While I am considering adding more autodiscovery plugin, it's not my current priority
Multiple manifests
Another attempt was to maintain multiple manifest, one for patch update, minor,etc..
with different CI jobs, like weekly, daily,etc.
This doesnt' scale very well as the number of manifest grow.
Quickly I gave up but I recently realized that using OCI registry, it could be a good way to publish and reuse policies
cfr https://github.com/updatecli/updatecli/issues/1600
so something like
Single
sources:
  default:
    kind: yaml
    spec:
      file: /tmp/data.yaml
      key: $.version
    transformers:
      - trimprefix: v
      - findsubmatch:
          # uncomment if we want patch version update
          #pattern: '^(\d*).(\d*)'
          pattern: '^(\d*)'
          captureindex: 0

  updatecli:
    kind: githubrelease
    dependson:
      - default
    spec:
      owner: updatecli
      repository: updatecli
      token: '{{ requiredEnv "GITHUB_TOKEN" }}'
      versionfilter:
        kind: semver
        pattern: '{{ source "default" }}.x'

conditions:
  default:
    kind: dockerimage
    sourceid: updatecli
    spec:
      image: ghcr.io/updatecli/updatecli

targets:
  default:
    kind: yaml
    sourceid: updatecli
    spec:
      file: /tmp/data.yaml
      key: $.version

could be made more generic by leveraging go template "range"
and then we would just have to provide the values file.
Support for OCI registry is very promising and will come soon.
I'll keep the follow up later today


Login pipeline trying to create new user when user exists

Hi All
I'm having a problem that when I login occationally the pipeline will attempt to create a new user even though I've already got a previously created and authenticated user stored.
Is there some steps I'm missing when doing this?
Here is my pipeline:
SOCIAL_AUTH_PIPELINE = (
'social.pipeline.social_auth.social_details',
'social.pipeline.social_auth.social_uid',
'social.pipeline.social_auth.auth_allowed',
'social.pipeline.social_auth.social_user',
'social.pipeline.user.get_username',
'social.pipeline.user.create_user',
'social.pipeline.social_auth.associate_user',
'social.pipeline.social_auth.load_extra_data',
'social.pipeline.user.user_details',
'custom_user.twitter_pipeline.get_auths',
)
the last stage is only to get the oauth_secret etc for posting to twitter etc.
Thanks in advance!

Did you upgrade from the django-social-auth app? I'm doing that for an application that's new to me and seeing the same issue, but now I notice the migration for this app contains an additional table social_auth_code that may do the connections.

I need more details to debug this one, what social site are you using? Could you define steps to reproduce?

This was the item you helped me with. Just needed to define:
SOCIAL_AUTH_PIPELINE = (
    'social.pipeline.social_auth.social_details',
    'social.pipeline.social_auth.social_uid',
    'social.pipeline.social_auth.auth_allowed',
    'social.pipeline.social_auth.social_user',
    'social.pipeline.user.get_username',
    'social.pipeline.social_auth.associate_by_email',
    'social.pipeline.user.create_user',
    'social.pipeline.social_auth.associate_user',
    'social.pipeline.social_auth.load_extra_data',
    'social.pipeline.user.user_details'
)

for Google+ anyway.


Add troubleshooting information

The following would have saved me several hours...

That’s very useful, thank you very much!
Related issue: #221.

Fixed spelling, added note about windows. For me (I'm on windows) it made many of my icons have corrupted paths, but that may be down to something else.

:+1:


Getting error  Missing required property: defaultLanguageTag

Describe the bug
I have created a Teams App and trying to run gulp ngrok-serve command.
This is giving error as shown below:

[23:13:30] Starting 'ngrok-serve'...
[23:13:30] Starting 'start-ngrok'...
[23:13:30] [NGROK] starting ngrok...
[23:13:36] [NGROK] Url: https://4dfe1e9f.ngrok.io
[23:13:36] [NGROK] You have been assigned a random ngrok URL that will only be available for this session. You wil need to re-upload the Teams manifest next time you run this command.
[23:13:36] [NGROK] HOSTNAME: 4dfe1e9f.ngrok.io
[23:13:36] Finished 'start-ngrok' after 6.18 s
[23:13:36] Starting 'manifest'...
[23:13:36] Starting 'validate-manifest'...
[23:13:36] Starting 'generate-manifest'...
[23:13:36] Finished 'generate-manifest' after 30 ms
[23:13:36] Starting 'schema-validation'...
[23:13:36] Using manifest schema 1.5
[23:13:38] 'schema-validation' errored after 2.05 s
[23:13:38] Error in plugin "validate-manifest"
Message:
Missing required property: defaultLanguageTag
[23:13:38] 'validate-manifest' errored after 2.09 s
[23:13:38] 'manifest' errored after 2.09 s
[23:13:38] 'ngrok-serve' errored after 8.27 s
To Reproduce
Steps to reproduce the behavior:

Follow https://camerondwyer.com/2019/09/09/how-to-create-a-microsoft-teams-messaging-extension-pop-up-dialog-with-a-custom-ui/ to create Teams App.
Run gulp ngrok-serve
See error

This is a normal error and will happen if you have an invalid schema. like you have the localizationInfo set in your manifest.json without the defaultLanguageTag.

What version of the generator do you have now?
What version of the generator was used to create the project, if it was different than the one now?

I am using 2.11.1 and it is the same version on which project was created.

Removed localizationinfo since dont need localization in App and now its working. Thanks.

Please share your manifest.json. DId you run the generator multiple versions? Did you select to use localization/translations?

I created new Teams App with below configuration

Once it's created run gulp ngrok-serve and got below error:

manifest.json
{
"$schema": "https://developer.microsoft.com/en-us/json-schemas/teams/v1.5/MicrosoftTeams.schema.json",
"manifestVersion": "1.5",
"id": "0780b450-de86-11e9-b5d6-e5dd87598b55",
"version": "0.0.1",
"packageName": "test",
"developer": {
"name": "Avanade",
"websiteUrl": "https://ca9c2e6a.ngrok.io",
"privacyUrl": "https://ca9c2e6a.ngrok.io/privacy.html",
"termsOfUseUrl": "https://ca9c2e6a.ngrok.io/tou.html"
},
"name": {
"short": "Teams Hackathon",
"full": "Teams Hackathon"
},
"description": {
"short": "TODO: add short description here",
"full": "TODO: add full description here"
},
"icons": {
"outline": "icon-outline.png",
"color": "icon-color.png"
},
"accentColor": "#D85028",
"configurableTabs": [],
"staticTabs": [
{
"entityId": "default-data",
"name": "Teams Hackathon",
"contentUrl": "https://ca9c2e6a.ngrok.io/teamsHackathonTab/",
"scopes": [
"personal"
]
}
],
"bots": [],
"connectors": [],
"composeExtensions": [],
"permissions": [
"identity",
"messageTeamMembers"
],
"validDomains": [
"ca9c2e6a.ngrok.io"
],
"localizationInfo": {}
}

As you can see in the screenshot,  you are running 2.11.0-preview and not 2.11.1 where this is fixed (see #81)