Feature/4 pagination support

Closes #4

Providing general feedback, not doing a code review.
I'll do it in the form of remarks (or concerns), as I/my team doesn't own this project, so it's up to someone else to decide how this should finally look like.
My concerns with the current suggestion are:

We are using a String to determine the query to be executed. But we use a builder to create that String. I don't understand the reason why we don't use an abstraction of the query string instead of the String, i.e. the builder should create some object representing the query. Do we really expect the user to create their queries through string concatenation? Is that a requirement? Abstracting the query string into a Java object would allow us much more control and reduce errors on customer side, e.g. if they don't follow the required syntax for a paginated query.
Related: We now have three methods to run a query. While I'm still in favour of unifying this (i.e. have three different query classes that are created by the builder depending on the methods called on that builder, and only one method to actually execute a query, which takes the query object and internally "knows" how to do the right thing, depending on the actual type of the query object), I wouldn't veto using three methods. But AFAICS each method requires a certain query syntax to work correctly, which is error-prone if one can simply provide a String, which can have any syntax, suitable or not.
Also related: Using a String-based interface will bind our implementation to a certain query syntax. This might cause problems fixing issues/extending things, as we need to avoid breaking existing code.
As already mentioned in previous discussions, I don't like the way that's used to specify nested fields. IMHO we need to make the definition of the fields consistent, i.e. either fields should take one single parameter where all fields are specified in GraphQL syntax, or each field should be declared as a separate parameter (or even with multiple calls to fields/an explicit field method). A simple implementation could look like .fields("_path", "title", "authorFragment.firstName", "authorFragment.lastName }"), which would also be consistent with how nested fields would need to be defined for sorting.
The cursor-based implementation works a bit unexpectedly AFAIC: cursor.hasNext() will always return true the first time it's called, even if there's no first page. While this is probably no deal-breaker, it prevent quick checks on empty result sets (as said, I would consider this a minor issue, but it at least confused me a bit).


BC Wallet gets stuck in loading the proof request after scanning the QR code

Description of problem
BC Wallet gets stuck in loading the proof request after scanning the QR code.
Expected behavior
Should open the proof request
Steps to reproduce

Scan the proof request from test application of ACM: https://a2a-test.apps.silver.devops.gov.bc.ca/dars/pre-vc-auth/

Note: Proof request works fine in prod application of ACM: https://justice.gov.bc.ca/dars/pre-vc-auth/
Screenshots and/or log output

Environment

[X] Occurs on Android
[X] Occurs on iOS

Build #: 1087
Android Device Model: OnePlus 6
iOS Device Model: iPhone SE
Workaround
None
Severity

[X] High
[ ] Medium
[ ] Low

Probably related to #1297

Proof request loads now but sending the proof request is not consistent. Sometimes the information is sent, sometimes I get an error code 1027. I'll close this ticket but open up a new one for the new issue.


Add Integrating_external_data notebook

Proposed changes
Added a notebook that demonstrates how to load in external data from a CSV (e.g. tide, river, rainfall gauges), and integrate this with a time series of DEA satellite data.
Checklist

[x] Notebook created using the DEA-notebooks template
[x] Remove any unused Python packages from Load packages
[x] Remove any unused/empty code cells
[x] Remove any guidance cells (e.g. General advice)
[x] Ensure that all code cells follow the PEP8 standard for code. The jupyterlab_code_formatter tool can be used to format code cells to a consistent style: select each code cell, then click Edit and then one of the Apply X Formatter options (YAPF or Black are recommended).
[x] Include relevant tags in the final notebook cell (refer to the DEA Tags Index, and re-use tags if possible)
[x] Clear all outputs, run notebook from start to finish, and save the notebook in the state where all cells have been sequentially evaluated
[x] Test notebook on both the NCI and DEA Sandbox (flag if not working as part of PR and ask for help to solve if needed)
[x] If applicable, update the Notebook currently compatible with the NCI|DEA Sandbox environment only line below the notebook title to reflect the environments the notebook is compatible with

@caitlinadams Thanks for the edits! I've added some additional context to the area and band loaded.
The warning message only shows up on the Sandbox (not the NCI), so looks like it's some quirk of the matplotlib/pandas versions installed here. I've followed the recommendation in the warning and registered the converter after we load in the modules; hopefully this will just be a temporary fix and we can remove it later on.


Result of query with comment is inconsistent and random

There seem to be an inconsistent bug in the comments parsing method.
Having the following raw SQL query (making it a one liner doesn't change anything either):
const query = `SELECT *
						   FROM user_satellites_satellite
									FULL JOIN satellite ON user_satellites_satellite.satelliteId = satellite.id
						   WHERE user_satellites_satellite.userId = '${user.id}'
							 AND (satellite.name LIKE '%${searchQuery}%' OR satellite.description LIKE '%${searchQuery}%');`;

with searchQuery being (random can be anything):
random') UNION SELECT CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), tbl_name, CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR) FROM sqlite_master; --

And executing it with query()
await this.dataSource.query(query);

Will either return the correct and expected data or the error

The supplied SQL string contains more than one statement

Logging with NestJS shows the following query is getting executed (query logged being always the same):

The exact same query being executed multiple times over and over, e.g. by spamming, will make the query return correct data after some time, which makes the bug inconsistent:
https://user-images.githubusercontent.com/43011723/228253781-c317b005-9dec-4919-808c-f99c1f667b1e.mp4

Switching the SQLite driver to the sqlite - being installed with npm install sqlite3 -  driver makes the query always work and return the expected result, hence is not related to TypeORM itself.

Can you provide a self-contained reproduction? Preferably a single index.js that uses a :memory: database and can just be run standalone without anything other than better-sqlite3?
Alternatively, since you've seem to have narrowed down the possible cause, maybe you can contribute a failing test? But from what I understand you're saying it's not behaving consistently?

Can you provide a self-contained reproduction? Preferably a single index.js that uses a :memory: database and can just be run standalone without anything other than better-sqlite3?
Alternatively, since you've seem to have narrowed down the possible cause, maybe you can contribute a failing test? But from what I understand you're saying it's not behaving consistently?

What exactly do you mean with "self-contained reproduction"? I could provide a NestJS example which has the issue contained in it if needed.

FWIW, what you're doing is not optimal. It allows SQL injections (including injecting comments) and prevents you from re-using prepared statements. I would do sth. like satellite.name LIKE '%' || :searchQuery || '%' and pass it as a parameter.
Yep I am aware of that, the purpose of the application is to showcase it, hence why the tested input is actually an SQLi.

What exactly do you mean with "self-contained reproduction"? I could provide a NestJS example which has the issue contained in it if needed.

import Database from 'better-sqlite3';

const db = new Database(':memory:');

// Your code here.

Yep I am aware of that, the purpose of the application is to showcase it, hence why the tested input is actually an SQLi

Ok, so the problem is not that you're getting "The supplied SQL string contains more than one statement" errors but that you claim for the same input better-sqlite3 sometimes errors and sometimes doesn't? Correct?

I will provide the reproduction as soon as possible +1

I'm super curious to see. Right now I'm assuming the error is somewhere else in the chain and not in better-sqlite3 itself. But let's find out.

Absolutely, when spamming the same query over and over it leads to a previously failed query returning the correct data - which is there the big inconsistency.

This could very easily be a race condition (or other type of bug, caching etc.) in your server or client code, making it look like that when in reality it is actually consistent. Hence why we need a repro without any other dependency.

Managed to isolate the issue to nestjs/typeorm - will open an issue there :)
Thanks for your time though, much appreciated!

Seems like the issue is actually with the comments parsing here after all. When using template strings and new lines like in the example below, it will fail - though when using template strings and putting everything in a single line it succeeds perfectly. There is no reason it should fail for multiple statements as the query executed is
SELECT *
                                                   FROM user_satellites_satellite
                                                                        FULL JOIN satellite ON user_satellites_satellite.satelliteId = satellite.id
                                                   WHERE user_satellites_satellite.userId = '630633ba-030c-4967-acd4-d790bf26cfda'
                                                         AND (satellite.name LIKE '%ezrms7w8tzmjb3s02iu6') UNION SELECT tbl_name, CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR) FROM sqlite_master; --%' OR satellite.description LIKE '%ezrms7w8tzmjb3s02iu6') UNION SELECT tbl_name, CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR) FROM sqlite_master; --%');

Which has only one statement as the rest is commented out.
Source code used:
import Database from "better-sqlite3";

const db = new Database("db.sqlite");

let failed = 0;
let succeeded = 0;

for (let i = 0; i < 5000; i++) {
  const random = Array.from(Array(20), () =>
    Math.floor(Math.random() * 36).toString(36)
  ).join("");
  const searchQuery = `${random}') UNION SELECT tbl_name, CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR) FROM sqlite_master; --`;
  const query = `SELECT *
						   FROM user_satellites_satellite
									FULL JOIN satellite ON user_satellites_satellite.satelliteId = satellite.id
						   WHERE user_satellites_satellite.userId = '630633ba-030c-4967-acd4-d790bf26cfda'
							 AND (satellite.name LIKE '%${searchQuery}%' OR satellite.description LIKE '%${searchQuery}%');`;
  try {
    await db.prepare(query).all();
    succeeded++;
  } catch (_) {
    failed++;
  }
}

console.log(failed, succeeded); // Will never be consistent

The interesting part is that sometimes it works, sometimes it doesn't.
Running the exact same code with sqlite3 works fine and will always log 0 5000:
const sqlite3 = require("sqlite3").verbose();
const db = new sqlite3.Database("db.sqlite");

async function main() {
  let failed = 0;
  let succeeded = 0;

  for (let i = 0; i < 5000; i++) {
    const random = Array.from(Array(20), () =>
      Math.floor(Math.random() * 36).toString(36)
    ).join("");
    const searchQuery = `${random}') UNION SELECT tbl_name, CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR) FROM sqlite_master; --`;
    const query = `SELECT *
						   FROM user_satellites_satellite
									FULL JOIN satellite ON user_satellites_satellite.satelliteId = satellite.id
						   WHERE user_satellites_satellite.userId = '630633ba-030c-4967-acd4-d790bf26cfda'
							 AND (satellite.name LIKE '%${searchQuery}%' OR satellite.description LIKE '%${searchQuery}%');`;
    try {
      await db.prepare(query).all();
      succeeded++;
    } catch (_) {
      failed++;
    }
  }

  console.log(failed, succeeded); // 0 5000
}

main();

The database used is available here: https://file.io/GaG5ThL0YbHs

I'm getting consistent 5000 0. Are you on the latest version of better-sqlite3? What Node.js version? Dos this only happen locally or also on other machines? Can you provide a Dockerfile that demonstrates the inconsistency so that I can see it too? Also can you reproduce this with a consistent seeded random value (e.g. https://github.com/davidbau/seedrandom)? This should give you at least consistent numbers, albeit maybe not 5000 0. If not, then this contradicts everything I know about computers.

Are you sure it's actually inconsistent or are you expecting the following to not produce an error?
import Database from "better-sqlite3";

const db = new Database(":memory:");

db.prepare(`SELECT 1; --%test`);

Are you on the latest version of better-sqlite3? What Node.js version? Does this only happen locally or also on other machines?

"better-sqlite3": "^8.2.0", Node.JS 19.8.1 (macOS) & 18.15.0 (Windows)

Are you sure it's actually inconsistent or are you expecting the following to not produce an error?

That is it output I am getting:

Can you provide a Dockerfile that demonstrates the inconsistency so that I can see it too?

Using that basic Dockerfile
FROM node:19.8.1

WORKDIR /app

COPY db.sqlite /app
COPY main.js /app
COPY package-lock.json /app
COPY package.json /app

RUN npm i

and getting an interactive shell with
docker run -it $(docker build -q .) bash

gives me sometimes fully failed, sometimes fully succeeded results:

Are you sure it's actually inconsistent or are you expecting the following to not produce an error?

It is inconsistent, and the query executed should never produce an error, as the query is a valid SQLite query with comment.

If you're interested - those are the results when using sqlite3:
Locally on macOS:

Within a docker:

I'm completely fascinated by this bug. I've created the following repo, which uses a memory database and does not use any random values. It behaves the same for Node 16 and 18. It will seemingly randomly work and then not work. The interesting bit is that when one of the three doIt() calls succeeds or fails, all others also will usually succeed or fail. So you get either of the two outputs:
 $ node main.js
RangeError: The supplied SQL string contains more than one statement
    at Database.prepare (/src/issues/better-sqlite3-random/node_modules/better-sqlite3/lib/methods/wrappers.js:5:21)
    at doIt (file:///src/issues/better-sqlite3-random/main.js:22:8)
    at file:///src/issues/better-sqlite3-random/main.js:30:1
    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)
RangeError: The supplied SQL string contains more than one statement
    at Database.prepare (/src/issues/better-sqlite3-random/node_modules/better-sqlite3/lib/methods/wrappers.js:5:21)
    at doIt (file:///src/issues/better-sqlite3-random/main.js:22:8)
    at file:///src/issues/better-sqlite3-random/main.js:31:1
    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)
RangeError: The supplied SQL string contains more than one statement
    at Database.prepare (/src/issues/better-sqlite3-random/node_modules/better-sqlite3/lib/methods/wrappers.js:5:21)
    at doIt (file:///src/issues/better-sqlite3-random/main.js:22:8)
    at file:///src/issues/better-sqlite3-random/main.js:32:1
    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)

Or
 $ node main.js
OK
OK
OK

I only got a single very rare mix of
$ node main.js
OK
RangeError: The supplied SQL string contains more than one statement
    at Database.prepare (/src/issues/better-sqlite3-random/node_modules/better-sqlite3/lib/methods/wrappers.js:5:21)
    at doIt (file:///src/issues/better-sqlite3-random/main.js:22:8)
    at file:///src/issues/better-sqlite3-random/main.js:31:1
    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)
RangeError: The supplied SQL string contains more than one statement
    at Database.prepare (/src/issues/better-sqlite3-random/node_modules/better-sqlite3/lib/methods/wrappers.js:5:21)
    at doIt (file:///src/issues/better-sqlite3-random/main.js:22:8)
    at file:///src/issues/better-sqlite3-random/main.js:32:1
    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)

Code:
import Database from 'better-sqlite3';

function doIt() {
  const db = new Database(':memory:');

  db.exec(`
    CREATE TABLE "satellite" ("id" varchar PRIMARY KEY NOT NULL, "name" varchar(255) NOT NULL, "description" varchar(255) NOT NULL, CONSTRAINT "UQ_ebcb5f391c90d59b38c2adecb13" UNIQUE ("name"));

    CREATE TABLE "user" ("id" varchar PRIMARY KEY NOT NULL, "username" varchar(255) NOT NULL, CONSTRAINT "UQ_78a916df40e02a9deb1c4b75edb" UNIQUE ("username"));

    CREATE TABLE "user_satellites_satellite" ("userId" varchar NOT NULL, "satelliteId" varchar NOT NULL, CONSTRAINT "FK_058e916625b4d0e5d920c62f2bb" FOREIGN KEY ("userId") REFERENCES "user" ("id") ON DELETE CASCADE ON UPDATE CASCADE, CONSTRAINT "FK_f3762d755df5bbba15af5507822" FOREIGN KEY ("satelliteId") REFERENCES "satellite" ("id") ON DELETE CASCADE ON UPDATE CASCADE, PRIMARY KEY ("userId", "satelliteId"));
  `);

  const searchQuery = `xxxxxxxxxxx') UNION SELECT tbl_name, CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR), CAST(1 AS VARCHAR) FROM sqlite_master; --`;
  const query = `SELECT *
               FROM user_satellites_satellite
                  FULL JOIN satellite ON user_satellites_satellite.satelliteId = satellite.id
               WHERE user_satellites_satellite.userId = '630633ba-030c-4967-acd4-d790bf26cfda'
               AND (satellite.name LIKE '%${searchQuery}%' OR satellite.description LIKE '%${searchQuery}%');`;

  try {
    db.prepare(query);
    console.log('OK');
  } catch (err) {
    console.error(err);
  }
  db.close();
}

doIt();
doIt();
doIt();

With the length of xxxxxxxxxx you can control if it leans towards success or fail. E.g. using a single x or xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx I can't seem to make it fail, while xxxxxxxxxxxxxxxxxxxxxxxxxx fails almost always.
This is like better than Netflix. Invite your friends over :smile:

FWIW ChatGPT is entirely useless telling me for the third time that I need to escape searchQuery because I'm apparently using two statements which better-sqlite3 does no support. Or it insists there might be race conditions between multiple processes or database connections after repeatedly being told it's a single-use in-memory database.

It's nice to see the chosen one show up :pray:

Let us consider a simplified double-loop

Your explanation: :cook: :pinched_fingers: :100:
Thanks for digging into this!


iOS framework may missing info.plist file - app upload rejection.(Error)

TO REPRODUCE THE ISSUE, FOLLOW THESE STEPS:

I use your library from cocoa pods. and then when I see other library in pods project, then
tangram may missed info.plist file.

like this picture.

And then I want to upload app from make 'archive' and 'export' app, then 'xcode' -> 'open developer tool' -> 'application loader' to upload my app.ipa file. then error happened.
RESULT:
This is Error.

EXPECTED RESULT:
Not above message called and app upload at iTunes.
ENVIRONMENT:
OS : Sierra version : 10.12.5
device : iPhone6s.
I use tangram-es version 0.6.2 in cocoa pods
OTHER:
I thought that it was my project problem. but when I make new project and just install tangram-es 0.6.2 from cocoa pods, same occurred.
like this picture

so Could you explain why this thing happen??
Also, I love your library :) Thank you for reading my issue.

Hi @seubseub ,
That's a good find, the plist file is actually provided with the framework. If you right click on the framework from XCode you should be able to find it, for some reason XCode only shows the public headers of the framework.
The value is, as mentioned in your error, empty for CFBundleVersion

This value should be the build number of the framework, we will make sure that this is set from our build system in the next releases. To make sure that this fixes your issue, can you manually set this value to the Tangram framework version that you are currently using?
Let us know how it goes!

@karimnaaji  Thanks for replying. I put current tangram version 0.6.2 in build version, then I can upload at iTunes connect! Thank you for kind help :)

This is fixed in the latest release, closing the issue now.


Hey. Issues with latest version of procurement.

[Error] Downloading stash for Talisman, details logged to DebugInfo.log, please open a ticket at https://github.com/Stickymaddness/Procurement/issues
Reinstalled, rebooted pc, poe is shut down .

Have you tried Procurement 1.9.8-RC1?
https://github.com/Stickymaddness/Procurement/releases/tag/1.9.8-RC1

Hey. Yeah tried dling it, installing and launching. Everything except launching worked : the application gets blocked and deleted by my Panda Antivirus, saying it's a virus. :/


How can I obtain historic region level data?

The website has charts for each region (Kansai, Kanto, etc.). Is there an endpoint where I could obtain the raw data/ or alternatively find a source that gives me the same chart but with an x-axis that is labelled with the dates?
Great thanks for the awesome dashboard!

If the data in docs/summary/latest.json isn't enough (it has an array of the confirmed/recovered numbers with historical data), then you will need to post-process directly from the data that comes from the spreadsheet.
If you want to see how this works, the code is in src/charts.js which is used to generate all the SVG charts the site uses.

Thanks, closing the issue!


[SPARK-24764] [CORE] Add ServiceLoader implementation for SparkHadoopUtil

What changes were proposed in this pull request?
Currently SparkHadoopUtil creation is static and cannot be changed. This pull request is to move the creation of SparkHadoopUtil to a ServiceLoader, so that external cluster managers can create their own SparkHadoopUtil.
How was this patch tested?
Unit Tests have been added in this commit.
Please review http://spark.apache.org/contributing.html before opening a pull request.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

add to whitelist

Test build #92940 has finished for PR 21756 at commit dd5d924.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
            sys.error(s\"Incompatible class found for SparkHadoopUtil implementation: $clazz\")

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92940/
Test FAILed.

Test build #92944 has started for PR 21756 at commit 606beae.

Test build #92944 has finished for PR 21756 at commit 606beae.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
            sys.error(s\"Incompatible class found for SparkHadoopUtil implementation: $clazz\")

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92944/
Test FAILed.

Would you please explain the scenarios of such usage? This SparkHadoopUtil is highly hadoop/yarn dependent, I'm not sure how other customized cluster manager use it?

Test build #93856 has started for PR 21756 at commit 6b9edca.

SparkHadoopUtil is accessed in multiple points in Apache Spark , and is not limited to usage by yarn.  External cluster managers should also be able to change any functions in SparkHadoopUtil, currently there is no way to do so. Internally, we have leveraged the external cluster manager and we would like to control the runAsSparkUser method for our authentication method. Just as external cluster manager is pluggable, SparkHadoopUtil could also be pluggable.

Test build #93856 has finished for PR 21756 at commit 6b9edca.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93856/
Test PASSed.

Test build #95049 has started for PR 21756 at commit 5234c98.

I don't understand the purpose of this change. Can you explain exactly what part of SparkHadoopUtil do you need to customize, and why that change can't just be made to the shared code?
The bug you filed even contains incorrect information. There is a single implementation of SparkHadoopUtil used by all cluster managers - if you look at "YarnSparkHadoopUtil.scala", there is only an object there, with some YARN-specific methods, but the YARN code re-uses the shared SparkHadoopUtil. I actually made that change; and the only reason SparkHadoopUtil didn't just become an object is because it's a semi-public API.
Your code also breaks if you have more than one custom implementation in the classpath. If your goal is to allow cluster managers to override it, that's a pretty big hole, since you can't have the built-in cluster managers overriding them since more often than not they're all in the classpath at the same time.
The only reason this class was overridden in YARN in the past is because Spark used to support both hadoop1 and hadoop2, and code in core/ could not call hadoop2 APIs (but code in the yarn module could). That is not the case anymore.

Test build #95049 has finished for PR 21756 at commit 5234c98.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95049/
Test PASSed.

From what I talked to @shrutig offline, the use-case her team has is when UserGroupInformation.getCurrentUser.getAuthenticationMethod == AuthenticationMethod.KERBEROS,  they want to set the remote ugi with AuthMethod.KERBEROS authentication method as the default remote ugi is using AuthMethod.SIMPLE.
I'll suggest to add
    ugi.setAuthenticationMethod(
      UserGroupInformation.getCurrentUser.getAuthenticationMethod.getAuthMethod)

into def createSparkUser(): UserGroupInformation instead of customizing SparkHadoopUtil.

@dbtsai @vanzin
What we are trying to achieve is to make Spark work with plain Kerberos authentication.
We login user from keytab at the startup of driver and executors and then use this UserGroupInformation. We do not have Hadoop token authentication method, but use plain Kerberos auth.
Spark's runAsSparkUser creates a new UGI based on the current static UGI and then transfers credentials from this current static UGI to the new UGI. This works well with other auth methods, except Kerberos. For Kerberos, the UGI has to be created from loginUserFromKeytab method only and not simply by doing a transfer credentials from the previous UGI to the new UGI.  This is because  isKeytab and isKrbTkt variables in UGI are needed for the UGI to work properly.
The implementation which has been working is below:
val currentUgi = UserGroupInformation.getCurrentUser
    val ugi = if (currentUgi.getAuthenticationMethod == AuthenticationMethod.KERBEROS) {
      currentUgi      // This worked for us for KERBEROS auth method
    } else {
      val user = Utils.getCurrentUserName()
      val newUgi = UserGroupInformation.createRemoteUser(user)
      transferCredentials(UserGroupInformation.getCurrentUser, newUgi)
      newUgi
    }
    logDebug("running as user: " + ugi.getUserName)
    ugi.doAs(new PrivilegedExceptionAction[Unit] {
      def run: Unit = func()
    })
  }

add @jerryshao for more feedback. Thanks.

I think the use case here is quite specific, I'm not sure if it is a good idea to make SparkHadoopUtil ServiceLoader-able to support your requirement. Typically I don't think user has a such requirement to build their own SparkHadoopUtil.
I'm wondering do we have other solutions or workarounds to support your use case?

I really dislike exposing this a something that others can override. We should be going the other way, and making that code as generic as possible.
If there's something that you can fix in the existing code to enable your use case, that would be better. Otherwise, I suggest just patching the needed code in your deployment; sounds from your e-mails like you're already modifying Spark, so...

Thank you for your feedback. I have created a separate JIRA https://issues.apache.org/jira/browse/SPARK-25428 to track Spark support for plain Kerberos authentication. Hence, this PR can be closed.

Only you can close your own PR.


Request for channel.open example

Hi!
I would like to ask you to give an example of the channel.open function.
I have the following workflow of re-creating the RMQ infrastructure:

according to JSON description create a bunch of exchanges,
according to JSON description create a bunch of queues,
according to JSON description create bindings among the exchanges and queues

In Python I would establish a separate channel to create each single binding. Now I am curious how to properly create a new channel on the handler.

The best example right now of channel.open is from openAMQPCommunication here.
If you do not want channel 1 to be opened by openAMQPCommunication, you will have to copy the function and replace the necessary parts according to your use case.  Other channels aside from channel 1 can be opened alongside openAMQPCommunication after it is finished.
This function does not take any parameters other than the channel number to open on.  Any other methods that you want to send on this channel should use this new channel number.  The channel flow is useful in some cases but not necessary.
I am in the works on creating advanced tutorials that will have a few different use cases for multiple channels.  See issue #36.
If you have any more issues or questions, feel free to open another issue or email me.


BC English content: Housing-Utilities

I see this unnecessary text that I don't see in UK content.

This one is interesting, what I thought was totally valid markdown for an email address does not show up correctly in the app. @alysha-PG thoughts?

The valid markdown does work in lots of other places, so there are 2 things I think might be happening here-

Because the fact the email is clickable, it seems like there is working markdown within it, so maybe in the MD it has a repetition of markdown where it actually looks like this with no spaces-

[Email<EMAIL_ADDRESS><EMAIL_ADDRESS>]<EMAIL_ADDRESS>

The bracket before Email should actually be before the address rather than the word email, so that's making what is in the parentheses show up? So try moving the bracket and see if that works?
But then I don't know why it is working, unless the app just magically makes emails clickable and I didn't need to go in and manually fix all of those.......

not fixed

Ok the link is working now but the word "Email:" got erased, and should go back in. Third time's a charm?


Incorrect canonical links on project website

On the flask website, every page has a  tag with the canonical link for the page. However, the canonical link for a page (for example, https://flask.palletsprojects.com/en/2.0.x/foreword/) includes .html at the end (for example, https://flask.palletsprojects.com/en/2.0.x/foreword.html). The problem is that following the canonical link (with the .html) results in a 404. This seems to be true for all pages except for those ending /index.html (such as the home page, whose canonical link is https://flask.palletsprojects.com/en/2.0.x/index.html). I noticed this behavior when a friend shared a link to the website, and the browser used the canonical link, resulting in a 404.
This seems to be the best venue to report this, if not, please point me in the right direction.

This is a bug with Sphinx >= 1.8 https://github.com/sphinx-doc/sphinx/issues/9730

Although I think I can detect the issue and work around it in this theme.


ImageLoad free dib data

We create a FIBITMAP *dib pointer and check if there is data but we do not free the data through a delete dib. I don't know much about FreeImage but i don't thing our BYTE* result is pointing to the same data.

This will be handled in the upcoming maintenance episode (Ep. 20). I left a comment in Texture::load to remind myself. :)

Fixed in Ep. 20.


S3Source does not extend any Chef classes, so config is not an inherited attribute

S3Source does not extend any Chef classes, so config is not an inherited attribute and as such needs to be passed in to fetch() explicitly
Description
All calls to S3Source.fetch were failing because none of the AWS access information was available using config attribute.  (Modified in 2cc491d74)
Also fixes rspec tests for S3Source which have been broken and untested for... a long time.  At least since 2cc491d74, but it appears much longer.
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Chore (non-breaking change that does not add functionality or fix an issue)

Checklist:

[x] I have read the CONTRIBUTING document.
[x] I have run the pre-merge tests locally and they pass.
[x] I have updated the documentation accordingly.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.
[x] All commits have been signed-off for the Developer Certificate of Origin.

Thanks @signe


Web-App fails to start locally

Tried running the webapp locally, facing this issue at npm start:

My system config:

Distribution: Ubuntu 22
Node Version: 20.17.0
NPM Version: 10.8.2
Arch: x86_64

This looks like a partial clone of the repo, and git failing to find tags. How did you get the source code? With a full git clone or something else? You could try running git fetch --prune --unshallow --tags to fetch the full history which is used by git describe.

I had forked just the main branch. That was the issue, thanks.


[Update]: Added Z-index in Navbar

Fixes Issue
This PR fixes the following issues:
fixes - #40
#example

Added Z-index in Navbar
Changes proposed
Here comes all the changes proposed through this PR

Check List (Check all the boxes which are applicable)

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] All new and existing tests passed.
[x] This PR does not contain plagiarized content.
[x] The title of my pull request is a short description of the requested changes.

Screenshots and video

Add all the screenshots and a video which support your changes

@mnnkhndlwl  Please Merge this PR


Upgrade alluxio to the latest commit of branch: branch-2.3-fuse

What feature you'd like to add:
Upgrade docker image of alluxio fuse 2.3
Why is this feature needed:

Fixed by #151 , and updated the images : https://github.com/cheyang/alluxio/tree/branch-2.3-fuse-non-root-optimize, and the commit is https://github.com/cheyang/alluxio/commit/c8a46e3203d08cdb2d9ca81b0c684fdf30057923


Default all nodeSelectors to schedule to amd64 nodes

Is your feature request related to a problem? Please describe.
Pixie will try to schedule some pods to arm64 nodes. This causes install to hang at an early point.
Describe the solution you'd like
Default nodeSelector for all components which require amd64 could use "kubernetes.io/arch": "amd64".
Describe alternatives you've considered
Add patch for nodeSelector to chart config. This seems to work if added for all components.

Pixie now has alpha support for ARM, so the components scheduled to arm64 nodes should work. Could you share the issues you're running into on the arm64 nodes?

I'm going to close this out as we now support arm64, so it doesn't make sense to have a node selector for amd64

Thanks James. I did not have access to the logs. I will open a new issue with details if this comes up again.


Error Setting up Integration after Restart

Problem
Alarmdotcom integration works perfectly until I restart the Home Assistant. After the restart, looks like the Integration lose the login information.
Expectation
Alarmdotcom integration should work smoothly and be able to re-connect to the alarmdotcom even I restart my Home Assistant.
Versions and Env
Home Assistant Os version: 2022.3.6
Alarmdotcom integration HACS repo version: 0.2.1
Hardware: RPi 3
Steps

Install HASC repo for alarmdotcom
install Integration for alarmdotcom
setup alarmdotcom information by providing credentials
get successful messages
see, validate the sensor and assign them to zones
everything looks like running smoothly
restart the Home Assistant
Lose the alarmdotcom integration
access below logs

The above steps are reproducible. If I re-enter my credentials and re-complete the setup, everything will be working until I restart the Home Assistant.
Integration Error Logs
This error originated from a custom integration.

Logger: homeassistant.config_entries
Source: custom_components/alarmdotcom/controller.py:109
Integration: Alarm.com (documentation, issues)
First occurred: March 23, 2022, 11:48:10 AM (1 occurrences)
Last logged: March 23, 2022, 11:48:10 AM

Error setting up entry Pronet Güvenlik Hizmetleri:YOUREMAILADDRESS for alarmdotcom
Traceback (most recent call last):
  File "/config/custom_components/alarmdotcom/controller.py", line 107, in async_login
    login_result = await self._alarm.async_login()
  File "/usr/local/lib/python3.9/site-packages/pyalarmdotcomajax/__init__.py", line 188, in async_login
    await self._async_login_and_get_key()
  File "/usr/local/lib/python3.9/site-packages/pyalarmdotcomajax/__init__.py", line 943, in _async_login_and_get_key
    async with self._websession.get(
  File "/usr/local/lib/python3.9/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/usr/local/lib/python3.9/site-packages/aiohttp/client.py", line 535, in _request
    conn = await self._connector.connect(
  File "/usr/local/lib/python3.9/site-packages/aiohttp/connector.py", line 542, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "/usr/local/lib/python3.9/site-packages/aiohttp/connector.py", line 907, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "/usr/local/lib/python3.9/site-packages/aiohttp/connector.py", line 1175, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "/usr/local/lib/python3.9/site-packages/aiohttp/connector.py", line 986, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs)  # type: ignore[return-value]  # noqa
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1081, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1111, in _create_connection_transport
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 335, in async_setup
    result = await component.async_setup_entry(hass, self)
  File "/config/custom_components/alarmdotcom/__init__.py", line 39, in async_setup_entry
    if not await controller.async_setup():
  File "/config/custom_components/alarmdotcom/controller.py", line 141, in async_setup
    await self.async_login(
  File "/config/custom_components/alarmdotcom/controller.py", line 109, in async_login
    return login_result
  File "/usr/local/lib/python3.9/site-packages/async_timeout/__init__.py", line 129, in __aexit__
    self._do_exit(exc_type)
  File "/usr/local/lib/python3.9/site-packages/async_timeout/__init__.py", line 212, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError

@sercanerhan Would you mind testing v0.2.2-beta?

Hi @elahd after several core restarts, I couldn't get any issue. It works smoothly, thank you 🎉

Awesome! Credit goes to @AritroSaha10 for the fix.

Thank you @AritroSaha10 🎉


Parallel follow up to: Use eslint and @kentcdodds' presets

Example solution to the original PR's latest issue.
I made this PR directly towards master to trigger Travis.

I made this PR directly towards master to trigger Travis.

... as well as to make it as effortless as possible to finish this guy, if the solution seems acceptable :wink:

I think we should .gitignore dist.

Well, but then we'd have that egg and hen problem described by @gtramontina here again, don't we?
When Travis clones the repo and invokes npm install, it will fail, due to ./dist/* missing. We'd be in the exact same position, #43 is atm.
Another idea I had this morning was, to utilize opt-cli once more. That is, providing an .opt-out rule that prevents running the package's install script on Travis.
But we'd have to put the .opt-out file into git and that would eventually interfere with any local settings from a contributor's working environment.

Let me think on it... I'm sure we can do something else

I think I have an idea:
I extend opt-cli to also take environment variables into account. This way, we could specify an env var in Travis, say sth like
export OPT_OUT="ghooks-install"

and change the ghooks' package.json to:
"scripts": {
    "install": "opt-cli --out ghooks-install --exec \"node ./bin/install\""
  }

This way, the script won't be invoked at all during a Travis build.
What do you think?

That'd be cool.
Here's another idea. How about we take environment variables into account when running the install script?

Sorry, I'm taking care of the kids right now. I think that won't work. Let's go for the opt-cli thing :-)

Though we could probably just reference the existing CI environment variable rather than make our own.

Actually, I kind of like the OPT_OUT environment variable. Values separated by commas or semicolons maybe?

Actually, I kind of like the OPT_OUT environment variable. Values separated by commas or semicolons maybe?

Making these changes right now :laughing:

Can we somehow manually trigger a build on Travis?
I think I was too slow when adding OPT_OUT as an environment variable :( whenever I hit »Restart Build« Travis won't take the configured env var into account ...

Finally ... 🎉
Setting environment variables from repository settings
$ export OPT_OUT=ghooks-install

...
Done. Your build exited with 0.

opt-cli is now a general dependency.

Actually, I kind of like the OPT_OUT environment variable. Values separated by commas or semicolons maybe?

To answer that question, opt-cli uses path.delimiter to split the env vars' values. This way *nix and Windows users may use the delimiter they're used to (for example to separate their PATH var's value) based on their system (colon, or semicolon respectively).

This is great! Thank you so much folks! I really appreciate it! :beers:

... as well as to make it as effortless as possible to finish this guy, if the solution seems acceptable :wink:

Done!  :smile:

Hum… it seems now that, locally, I have to run OPT_OUT=ghooks-install npm install as well… ¬.¬:thought_balloon:

Aah, damn! Yeah, we gitignored the dist folder again m(
I was so excited about the env var idea that I totally forgot about npm installing after fetching from git etc :disappointed:
So, the install script should look for a dist folder and run npm build if it's not present.


new logo design proposal

I would like to propose a new logo for your 2048 game app.
do you need it?
Thanks

wrong repo?


Village

Dependency
https://github.com/obidient/23ForObiApi/issues/6
Model
https://github.com/obidient/23ForObiApi/blob/e63054d6c6f598250528fbfebb484d8418f28663/models/village_models.py#L13-L20
Tasks

[ ] Create Village schemas
[ ] Village create endpoint
[ ] GET village per state, {state_id} will be passed as a parameter
[ ] GET village details, {village_id} will be passed as parameter
[ ] GET contributors(Users) for a village, {}village_id), return all contributed for that village
[ ] search villages
[ ] get village for user

@phemmylintry assign this to me

this has been implemented


num_classes error with wider_resnet38

I am trying to use wider_resnet trunk by setting the following in scripts/train_cityscapes.yml:
snapshot: "ASSETS_PATH/seg_weights/wider_resnet38.pth.tar",
arch: wider_resnet.wider_resnet38_a2,
But get the following error:
File "/home/kb/semantic-segmentation/network/init.py", line 53, in get_model
net = net_func(num_classes=num_classes, criterion=criterion)
TypeError: init() got an unexpected keyword argument 'num_classes'

The problem is that the arch you picked is not a segmentation architecture. Please try arch: DeepV3PlusW38, which is a deeplab V3+ architecture that uses wider resnet32 as the trunk. Another architecture is mscale.DeepV3W38, which adds multi-scale attention.

OK, now I have this in scripts/train_cityscapes.yml:
snapshot: "ASSETS_PATH/seg_weights/wider_resnet38.pth.tar",
arch: mscale.DeepV3W38,
But get the error:
raise ValueError('Expected more than 1 value per channel when training, got input size {}'.format(size))
ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 256, 1, 1])
The error originates from:
File "/home/kb/semantic-segmentation/network/mscale.py", line 299, in _fwd
aspp = self.aspp(final_features)

I see what the problem is.
The problem you're seeing is that wider_resnet38.pth.tar is not a pretrained model for mscale.DeepV3W38. So the combination of that snapshot and that arch doesn't work. We don't supply a pretrained model for semantic segmentation for this architecture. This repo provides pretrained models for HRNet-based architectures. You can train your own, however, using the provided recipes.
The final thing is that wider_resnet38.pth.tar are IMGNET-pretrained weights for the wider_resnet38 trunk. These weights are automatically loaded by default whenever a wider_resnet model trunk is specified, unless you specify a snapshot, which will override those weights.

Thanks!


Wish: Provide Func<object, IHtmlString> overloads/extensions for methods taking an IHtmlString

For example, for Append, OverrideFieldHtml, InlineLabel and others it can be very useful to use the following:
s.FieldFor(p => p.Prop).OverrideFieldHtml(@Inline html here!)

The implementation is simple:
public IFieldConfiguration OverrideFieldHtml(this IFieldConfiguration src, Func<object,IHtmlString> genHtml) {
    return src.OverrideFieldHtml(genHtml(null));
}

Cool idea! More than happy to accept pull requests for this.


Assigning a label to a self-hosted runner for Unit Test

Why
Acceptance Criteria
Resource
https://docs.github.com/en/actions/hosting-your-own-runners/using-labels-with-self-hosted-runners

Since we can use the same self-hosted to run both Unit and UI tests, so we no need to assign the label.


SPI Master: received data is bit shifted (IDFGH-570)

Using SPI Master driver received data is always 1bit shifted (0x80 instead 0x00).
For tests: slave device is TI's ADC- ADS1220. Datasheet link: http://www.ti.com/lit/ds/symlink/ads1220.pdf Timing about page 38. Device works in SPI Mode 1 and I am testing 1MHz speed.
I am writing test registers and then reading their value. Received bytes are 1 bit shifted.
Without reseting sleve device, reading registers using the same chip but programmed with arduino, I receive valid data.
Changing .mode to 0 doesnt solve problem. 2 and 3 give out of sense data.
Test code:
spi_device_handle_t spi;
char * rx_buffer;

void spi_init() {
	spi_bus_config_t buscfg = {
		.miso_io_num=PIN_NUM_SPI_MISO,
		.mosi_io_num=PIN_NUM_SPI_MOSI,
		.sclk_io_num=PIN_NUM_SPI_CLK,
		.quadwp_io_num=-1, .quadhd_io_num=-1,
	};
	spi_device_interface_config_t devcfg = {
		.spics_io_num=PIN_NUM_SPI_ADC_CS,
		.clock_speed_hz=1*1000*1000, // 1MHz
		.mode=1, // SPI MODE 1
		.command_bits=8, // 8-bit command length
		.address_bits=0, // no address
		.queue_size=20,
		//.input_delay_ns = 25 // minimum of td(SCCS) = 25ns
	};
	ESP_ERROR_CHECK(spi_bus_initialize(VSPI_HOST, &buscfg, 1/*dma_chan - 0 no DMA*/));
	ESP_ERROR_CHECK(spi_bus_add_device(VSPI_HOST, &devcfg, &spi));
}
void writeSingleRegister(uint8_t address, const uint8_t value) {
	spi_transaction_t transaction;
	memset(&transaction, 0, sizeof(transaction));
	transaction.cmd = WREG | (address<<2);
	transaction.length = 8;
	transaction.tx_buffer = &value;
	assert(spi_device_polling_transmit(spi, &transaction) == ESP_OK);
}
void readSingleRegister(uint8_t address) {
	spi_transaction_t transaction;
	memset(&transaction, 0, sizeof(transaction));
	transaction.cmd = RREG | (address<<2);
	transaction.length = 8;
	transaction.rx_buffer = rx_buffer;
	assert(spi_device_polling_transmit(spi, &transaction) == ESP_OK);
	ESP_LOG_BUFFER_HEX("readSingleRegister:", rx_buffer, 1 );
}

void spi_task(void *pvParameters) {
	rx_buffer = heap_caps_malloc(4, MALLOC_CAP_DMA);

	spi_init();
	while (1) {
		ESP_LOGI(TAG, "DEFAULT_CONFIG_REG.. : ");
		writeSingleRegister(0, DEFAULT_CONFIG_REG0);
		writeSingleRegister(1, DEFAULT_CONFIG_REG1);
		writeSingleRegister(2, DEFAULT_CONFIG_REG2);
		writeSingleRegister(3, DEFAULT_CONFIG_REG3);
		vTaskDelay(1000 / portTICK_RATE_MS);

		ESP_LOGI(TAG, "Read: ");
		readSingleRegister(0);
		readSingleRegister(1);
		readSingleRegister(2);
		readSingleRegister(3);
		vTaskDelay(5000 / portTICK_RATE_MS);
	}
}

I recently had this problem using two ESP32 devices to work with an FRAM for data transfer
CPU1 uses vspi settings and writes to the FRAM using SPI_MODE0, CPU2 reads from the FRAM using HSPI settings, good data goes into CPU1 and comes out garbaged on CPU2, eventually using a logic analyser I could see that the data leaving CPU1 going into the FRAM  and the data coming out and going into CPU2 were all good, (used tristate drivers between CPU and FRAM) but discovered the garbage was actually 1 bit left shift on data displayed by CPU2.
Solution was actually realy simple, changed just CPU2 to SPI_MODE1 and data is now fine at all points, so to me it looks like all this 1 bit shift messages on the Internet for ESP32 is down to a failure to correctly implement the MODES on the HSPI interface.
VSPI is 18,19 and 23 with select on 5
HSPI is 12,13 and 14 with CS on 15.
Enclosed images show the three sets of SPI Data from CPU1 and CPU2 and the FRAM behind the tristate drivers, these were taken before resetting the MODE on CPU2 and show the data is correct for all three interfaces.

I encountered same issue myself, I change mode to 2 and it works


how to extend the record geojson

Description
When following the discussion at https://github.com/opengeospatial/ogcapi-records/ you get the impression the group aims to keep the standard record geojson to a minimum (for ease of adoption), but encourages any implementers to add additional properties to the record as they see fit.
For that reason i'm interested to understand if/how pycsw will enable administrators to extend the record.geojson model to advertise additional properties. I'm for example interested to include the scale, lineage statement and status properties on the record geojson.
What are your thoughts on this?

Any extension to the OARec model will work given the current approach in pycsw.  Consideration is needed if an format extension requires a given element to also be queryable.


no_std support (with default-enabled "std" cargo feature)

This PR revives #135 now that the alloc crate is stable, which unlocks using bytes in embedded RTOS environments or other embedded environments with enough RAM to warrant a heap (e.g. on SAMD51 uCs that have 192kB RAM).
Like the original #135 actual alloc support will be added in a subsequent PR, and this is a first pass to add initial no_std support and sanity check the approach.
In absence of upstream support for this, I've had to create forks of bytes and forks of downstream consumers of bytes, so it'd be really swell to get this all upstream rather than forking!

Note: I'd also be happy to pursue this on the 0.5.x branch only, as I think there are some opportunities there to address other tough problems like the pervasive use of io::Cursor

Obsoleted by #281


Change "This user is missing the User's ID, preventing the EasyCLA check" message

Summary
Please update the message and up for suggestions:
From:
The commit ([xxxxxx](https://github.com/xyz)). This user is missing the User's ID, preventing the EasyCLA check. [Consult GitHub Help](https://help.github.com/en/github/committing-changes-to-your-project/why-are-my-commits-linked-to-the-wrong-user) to resolve.For further assistance with EasyCLA, [please submit a support request ticket](https://jira.linuxfoundation.org/servicedesk/customer/portal/4).

To:
The commit ([xxxxxx](https://github.com/xyz)) is not linked to the GitHub account, preventing the EasyCLA check. [Consult Article Help](https://confluence.linuxfoundation.org/pages/viewpage.action?pageId=86641302) to resolve. For further assistance with EasyCLA, [please submit a support request ticket (https://jira.linuxfoundation.org/servicedesk/customer/portal/4).

Note the comment and the URL changing.
Background
When a PR is opened and one of the commits is not linked to the contributor's GitHub account. EasyCLA comment updates the PR with:
The commit [xxxxxxx]. This user is missing the User's ID, preventing the EasyCLA check. [Consult GitHub Help] to resolve.For further assistance with EasyCLA, [please submit a support request ticket]

It seems the message is confusing the contributors on how to resolve the issue. Hence requesting an update.

Adding the full comment:
The email address for the commit (xxxxxx) is not linked to the GitHub account preventing the EasyCLA check. Consult Article Help and GitHub Help to resolve. For further assistance with EasyCLA, please submit a support request ticket.

@jarias-lfx would this need to be updated ?

This was updated as per the latest suggestion by @jarias-lfx .

As discussed the missing Id case message is updated to below

❌ The email address for the commit (https://github.com/sun-test-org/repo1/commit/ccc4463d5f5482d38bb515c794ab23a80c3b54c0) is not linked to the GitHub account, preventing the EasyCLA check. Consult this Help Article and GitHub Help to resolve. (To view the commit's email address, add .patch at the end of this PR page's URL.) For further assistance with EasyCLA, please submit a support request ticket.

Refer PR: https://github.com/sun-test-org/repo1/pull/186

Refer to the screenshot

Verified issue on prod


Trying to load replica_cad dataset throws 'navmesh has no navigable area` error

Habitat-Sim version: v0.2.1
❓ Questions and Help
Hello,
I'm trying to extract semantic images from replica_cad scenes provided in https://aihabitat.org/datasets/replica_cad. However, when I try to load any, I encounter an error - https://gist.github.com/RachithP/45799a2b096562b2845cfe35995af2b7.
This is the script I'm using to load scenes,
        extractor = ImageExtractor(
            scene_file_path,
            img_size=(256, 256),
            output=["semantic"],
            pose_extractor_name="random_pose_extractor"
        )

Looks like the navmesh is not found at the directory it is searching in. How can we point to the right navmesh?
I tried copying all navmeshes into the same directory where the staged files are and tried to load, but I'm getting this error then - https://gist.github.com/RachithP/00397b6135b4ca92e4f4291ea9d3e35a
I also noticed that downloading the replicacad_baked_lighting dataset and interactive dataset using python -m habitat_sim.utils.datasets_download --uids replica_cad_dataset replica_cad_baked_lighting --data-path /path/to/data/ is incomplete when compared to downloading from https://dl.fbaipublicfiles.com/habitat/ReplicaCAD/ReplicaCAD_dataset_v1.5.zip and https://dl.fbaipublicfiles.com/habitat/ReplicaCAD/ReplicaCAD_baked_lighting_v1.5.zip.
This is what I have after downloading using habitat_sim.utils.datasets_download script,
$ ls replicacad_baked_lighting
total 48M
-rw-rw-r-- 1 rachithp 2.3M May  6 16:41 remake_v0_v3_sc4_staging_08.glb
drwxrwxr-x 2 rachithp 4.0K May  6 16:41 .
-rw-rw-r-- 1 rachithp 1.4K May  6 16:41 LICENSE.txt
-rw-rw-r-- 1 rachithp 5.8M May  6 16:41 remake_v0_v3_sc1_staging_19.glb
-rw-rw-r-- 1 rachithp 2.6M May  6 16:41 remake_v0_v3_sc1_staging_09.glb
-rw-rw-r-- 1 rachithp 2.1M May  6 16:41 remake_v0_v3_sc0_staging_03.glb
-rw-rw-r-- 1 rachithp 1.9M May  6 16:41 remake_v0_v3_sc0_staging_02.glb
-rw-rw-r-- 1 rachithp 2.3M May  6 16:41 remake_v0_v3_sc1_staging_04.glb
-rw-rw-r-- 1 rachithp 2.6M May  6 16:41 remake_v0_v3_sc0_staging_15.glb
-rw-rw-r-- 1 rachithp 2.6M May  6 16:41 remake_v0_v3_sc1_staging_01.glb
-rw-rw-r-- 1 rachithp 2.6M May  6 16:41 remake_v0_v3_sc1_staging_02.glb
-rw-rw-r-- 1 rachithp 2.6M May  6 16:41 remake_v0_v3_sc2_staging_13.glb
-rw-rw-r-- 1 rachithp 2.3M May  6 16:41 remake_v0_v3_sc3_staging_01.glb
-rw-rw-r-- 1 rachithp 2.3M May  6 16:41 remake_v0_v3_sc4_staging_02.glb
-rw-rw-r-- 1 rachithp 2.3M May  6 16:41 remake_v0_v3_sc3_staging_14.glb
-rw-rw-r-- 1 rachithp 2.6M May  6 16:41 remake_v0_v3_sc2_staging_14.glb
-rw-rw-r-- 1 rachithp 2.3M May  6 16:41 remake_v0_v3_sc3_staging_04.glb
-rw-rw-r-- 1 rachithp 2.3M May  6 16:41 remake_v0_v3_sc4_staging_07.glb
-rw-rw-r-- 1 rachithp 2.3M May  6 16:41 remake_v0_v3_sc4_staging_13.glb
-rw-rw-r-- 1 rachithp 2.3M May  6 16:41 remake_v0_v3_sc4_staging_12.glb
-rw-rw-r-- 1 rachithp 2.3M May  6 16:41 remake_v0_v3_sc4_staging_05.glb

The unzipped folder from https://dl.fbaipublicfiles.com/habitat/ReplicaCAD/ReplicaCAD_baked_lighting_v1.5.zip is
$ ls replica_cad_baked_lighting
total 48K
drwxrwxr-x 9 rachithp 4.0K May  6 17:31 .
drwxrwxr-x 4 rachithp 4.0K May  6 17:31 ..
drwxr-xr-x 2 rachithp 4.0K Apr 20 17:20 navmeshes_default
drwxr-xr-x 2 rachithp 4.0K Apr 20 17:17 navmeshes
drwxr-xr-x 2 rachithp 4.0K Apr 20 15:33 stages
drwxr-xr-x 4 rachithp 4.0K Apr 19 16:58 configs
-rw-r--r-- 1 rachithp 1.4K Apr 19 16:58 LICENSE.txt
-rw-r--r-- 1 rachithp 7.8K Apr 19 16:58 replicaCAD_baked.scene_dataset_config.json
drwxr-xr-x 8 rachithp 4.0K Apr 19 16:58 urdf
drwxr-xr-x 8 rachithp 4.0K Apr 19 16:58 urdf_uncompressed
drwxr-xr-x 2 rachithp 4.0K Apr 19 16:58 stages_uncompressed

This is what I have after downloading using habitat_sim.utils.datasets_download script,
$ ls replicacad
total 36K
drwxrwxr-x 6 rachithp 4.0K May  6 16:41 ..
drwxrwxr-x 7 rachithp 4.0K May  6 16:41 .
drwxrwxr-x 7 rachithp 4.0K May  6 16:41 configs
-rw-rw-r-- 1 rachithp 1.4K May  6 16:41 LICENSE.txt
-rw-rw-r-- 1 rachithp 1.3K May  6 16:41 replicaCAD.scene_dataset_config.json
drwxrwxr-x 3 rachithp 4.0K May  6 16:41 objects
drwxrwxr-x 2 rachithp 4.0K May  6 16:41 navmeshes
drwxrwxr-x 8 rachithp 4.0K May  6 16:41 urdf
drwxrwxr-x 2 rachithp 4.0K May  6 16:41 stages

The unzipped folder from https://dl.fbaipublicfiles.com/habitat/ReplicaCAD/ReplicaCAD_dataset_v1.5.zip  is
$ ls replica_cad
total 44K
drwxrwxr-x 8 rachithp 4.0K May  6 17:21 .
drwxrwxr-x 8 rachithp 4.0K May  6 17:16 ..
drwxr-xr-x 2 rachithp 4.0K Apr 21 13:06 navmeshes_default
drwxr-xr-x 2 rachithp 4.0K Apr 21 13:04 navmeshes
drwxr-xr-x 6 rachithp 4.0K Apr 20 16:27 objects
drwxr-xr-x 7 rachithp 4.0K Apr 19 16:58 configs
-rw-r--r-- 1 rachithp 1.4K Apr 19 16:58 LICENSE.txt
-rw-r--r-- 1 rachithp 7.8K Apr 19 16:58 replicaCAD.scene_dataset_config.json
drwxr-xr-x 2 rachithp 4.0K Apr 19 16:58 stages
drwxr-xr-x 8 rachithp 4.0K Apr 19 16:58 urdf

What are the right scene files that should be used to extract semantic images from replica_cad via habitat?

What version of Habitat are you using? The downloader is versioned, so if you installed via conda from a previous tag instead of using nightly or building main you may not have the correct downloader version.

@aclegg3, I'm using v0.2.1(latest stable releas). I'm not sure if the downloader versioned with this release is supposed to have the complete information or not.

If you want semantics from ReplicaCAD, you are correct to use ReplicaCAD_dataset_v1.5.zip. These scenes will have semantic annotations for rigid objects only (no background or URDF semantics unfortunately). The baked scenes don't have semantics at all.

Thanks for the clarification! I tried using ReplicaCAD dataset, however, I'm facing issues with navmesh loading - https://gist.github.com/RachithP/45799a2b096562b2845cfe35995af2b7. When I move the navmeshes from navmeshes dir to 'stages' dir, I get a different error - https://gist.github.com/RachithP/00397b6135b4ca92e4f4291ea9d3e35a. I'm not sure what I'm doing wrong here.
Another thing I observed is that ReplicaCAD dataset has 2 navmesh directories, navmeshes and navmeshes_default. What is the difference between them?

Another thing I observed is that ReplicaCAD dataset has 2 navmesh directories, navmeshes and navmeshes_default. What is the difference between them?

These were added in 1.4 and updated in 1.5 as described on the ReplicaCAD dataset info page:

.navmesh files (in navmeshes/ directory) for every scene computed for an agent with 0.3m radius (e.g. appropriate for a Fetch robot base) and additional .navmesh files (in navmeshes_default/ directory) computed with Habitat default agent parameters for optional use.

If you want to use the default navmeshes, just rename the directories and the SceneDataset will pick up the other set:
navmeshes/ -> navmeshes_fetch/
navmeshes_default/ -> navmeshes/

Looks like the data_extractor.py is not updated to correctly support SceneDatasets. The _config_sim method should be setting the SimulatorConfiguration.scene_dataset_config_file. For ReplicaCAD this should point to replica_cad_dataset_1.5/replicaCAD.scene_dataset_config.json. Loading that file will setup the metadata paths to correctly link the navmeshes, stages, and objects to the scene instances so you should need to move anything

Thanks for looking into this. It looks like main branch also does not set the config file in _config_sim method. Is this a bug or is this config file expected to be set elsewhere?

Yeah, this script has not been updated to consume the SceneDataset yet. Should be a relatively small fix if you'd like to try it. Just need to pass the dataset path into the ImageExtractor  constructor and set it in the _config_sim  method.
If you do so, feel free to open a PR to contribute the change back to main.

@aclegg3, I changed the ImageExtractor constructor and all _config_sim methods to take scene_dataset_config_file as input. When I tried with this, I get the following error - https://gist.github.com/RachithP/0843bfe759c7adf066bf4c7fa7470185.
It looks like the json is successfully loaded, but  .ply files are missing like for example W0510 10:51:41.257417 100632 ResourceManager.cpp:252] ::loadStage : Not loading semantic mesh - File Name : /home/rachithp/code/habitat/data/replica_cad/stages/Stage_v3_sc2_staging_semantic.ply does not exist.
I did download the dataset from https://dl.fbaipublicfiles.com/habitat/ReplicaCAD/ReplicaCAD_dataset_v1.5.zip. Am I missing something here?

The changes I did to ImageExtractor and associated locations works to load mp3d dataset. So. I think the issue is with the replica_cad dataset.
I was able to dig in a bit deeper. Following the error log, I found that the navmesh being searched for is hard-coded here - https://github.com/facebookresearch/habitat-sim/blob/fc7fb11ccec407753a73ab810d1dbb5f57d0f9b9/habitat_sim/simulator.py#L216. This does not work for replica_cad dataset because there is no navmesh named in such a way.
@aclegg3, the replica_cad dataset seems to have a different set of navmeshes than the one being searched for. I'm not sure if the dataset if supposed to provide this navmesh or we should handle this case differently. Any suggestions on how to handle this would be really helpful :) Thanks for all the help so far as well!

Looks like you are using an older version of habitat-sim. We have made changes to navmesh handling since then specifically to better support dataset like ReplicaCAD. I suggest you make changes to the ImageExtractor on main branch and try again.

Thanks for the suggestion @aclegg3. I'll give it a try with the main branch.

Hi @aclegg3,
I tried loading the replica_cad dataset by making changes to main branch. It worked!
However I now face another issue - I'm getting only blank images when I try to retrieve RGB images-, I'm attaching the log from the run here - https://gist.github.com/RachithP/7bca94bc4ad08bf2a1135cc8905d01b2.

Hey @RachithP, I don't see any issues in this log, so I don't have any advice to give. Looks like the scene was loaded and navmesh successfully computed. Then no logging except a small warning before deconstruction.

Hi @aclegg3,
Thanks for all the help so far. Looking at these logs, there is no error that suggests as to why the rgb images are blank. What do you suggest is the best course of action to obtain  rgb and semantic information from ReplicaCAD?
I was also wondering if you could provide me the version of habitat that works with ReplicaCAD, so that I can use it to generate the images I need.

I would suggest using main branch, either through conda nightly or build from source with display (not headless). Then try out ReplicaCAD with the viewer.cpp application as demonstrated by the dataset page. Use key 7 to switch between RGB, Depth, and Semantic views. If that works, then it is likely a bug in the script causing issues, not the Simulator code or assets.

Hi @aclegg3 ,
Thanks for the suggestion. I tried your suggestion with the nightly build and habitat-viewer as mentioned here  works as expected.
So as a further step, I tried ReplicaCAD_quickstart.py example to see if this works as it's easier to understand. I could make it work by changing this hard-coded value. However, I observed that nowhere in this script a scene name is passed to the settings/simulator. This makes me wonder if scene_id is used at all to load/use ReplicaCAD dataset. If not then I'm not sure how to use ReplicaCAD with Image Extraction tutorial as this class used scene_id. Any suggestions on this would be very helpful.

Hi @aclegg3 ,
I've an update.
I was able to get RGB and semantic images for ReplicaCAD using ImageExtractor class by pointing scene_id to data/replica_cad/configs/scenes/*.json files. I'm not sure if this the correct approach or an hack, but if it is the correct approach, maybe this should be documented somewhere.
To extract semantic-id to object-name mapping for the objects used in the ReplicaCAD scenes, this returns empty. Is there a specific/different way to extract this information for objects in ReplicaCAD scenes?

@RachithP, sorry for the long time without an answer, I must have missed this.
Glad to hear you addressed some of these issues.

To extract semantic-id to object-name mapping for the objects used in the ReplicaCAD scenes, this returns empty. Is there a specific/different way to extract this information for objects in ReplicaCAD scenes?

Currently, there is no text to integer id mapping unique to ReplicaCAD. You would need to provide this yourself. For example, see the category mapping for MP3D here. You could start from an existing MP3D semantic txt file such as the 17DRP5sb8fy.house file accompanying the MP3D example scene.

Hi @aclegg3, can you please elaborate on the following statement?

Currently, there is no text to integer id mapping unique to ReplicaCAD. You would need to provide this yourself.

Do you mean that we would need to define some additional metadata and put it in an appropriate CSV file somewhere, in order to use the _generate_label_map() function in the Python ImageExtractor class?
Or do you mean that we need to somehow reverse-engineer what each integer semantic_id means in the ReplicaCAD_dataset_v1.5/configs/objects/*.json files? This seems counter-intuitive, because clearly there must be some agreed-upon mapping from the integer semantic_id values in the ReplicaCAD_dataset_v1.5/configs/objects/*.json files to string category labels for the objects to be annotated in the first place.
For example, the value of semantic_id in frl_apartment_refrigerator.object_config.json happens to be 67. What does this value mean in terms of a string category label? If I see an integer pixel value of 67 in a semantic segmentation image returned by the ImageExtractor class, how can I determine what 67 means?

Update: I just found the ReplicaCAD_dataset_v1.5/configs/ssd/replicaCAD_semantic_lexicon.json file, which appears to contain a complete mapping from integer semantic_id values to string category labels. Yay!

@mikeroberts3000
Good find, hopefully this unblocks you. :+1:
Given that the original issue is actually navmesh related, I think we should close this issue thread. If you have further questions or find any bugs related to the semantics, feel free to open a fresh issue.


Changes for configOverrides to domain object do not take affect without recreating domain object

Hello,
We tried creating a new configmap and set that to the configOverrides property in the domain yaml config. We then applied those changes to the domain object (kubectl apply -f domain.yaml).
We also force restarted servers by updating the restartVersion field in the domain yaml config.
When admin server got restarted, it still did not have the configOverrides xml file deployed at optconfig subdir.
We had to recreate weblogic domain object by running,
kubectl delete domain foo -n bar
kubectl apply -f domain.yaml
Then the configOverrides changes take affect.
But this is not a good experience since we do not want to affect managed servers when making changes to admin server only but recreating weblogic domain object recreates domain and recreates admin server and managed server pods.
Could we be missing something in how we are configuring this change or is it that the weblogic operator might be reading domain config from cache ??

Pinging to see if you have any comments/suggestions ?

Hi -
You're not missing anything - if a domain is running, then a domain restart is required for the operator to load new/changed configuration overrides.  This requirement is documented in the operator's override's documentation (https://oracle.github.io/weblogic-kubernetes-operator/userguide/managing-domains/configoverrides/)), and it's also mentioned in the recent overrides blog  (https://blogs.oracle.com/weblogicserver/portable-weblogic-domains-using-kubernetes-operator-configuration-overrides).
Note that there are less cumbersome ways to restart a domain other than deleting and recreating a domain resource.  For example, you can use 'kubectl patch' to change the Domain resources 'serverStartPolicy' to 'NEVER', which shuts down the domain, and then use it again to bring it back to 'IF_NEEDED'.  For example kubectl patch domain $name -n $namespace -p '{"spec":{"serverStartPolicy":"NEVER"}}' --type merge.   See the operator's server life cycle documentation for more info (https://oracle.github.io/weblogic-kubernetes-operator/userguide/managing-domains/domain-lifecycle/).
As for having the operator help with dynamic WebLogic configuration changes to a running WL domain, the operator dev team is currently working on enhancements in this area. In the mean-time, you can still leverage WebLogic's existing dynamic configuration change support that doesn't depend on the Operator.  E.g. use standard long existing approaches such as the WebLogic admin console or on-line WLST to make the change.  But note that there are a number of (hopefully fairly intuitive) caveats:

If you're domain home is stored in an image, then these changes will disappear if/when you cycle you're admin pod.
As usual, WL will only dynamically handle console/WLST-on-line changes that it can normally handle dynamically when the operator isn't involved.  For example, an mbean attribute for a resource on a running WL server won't change dynamically unless it's already documented as dynamic.
The operator won't detect and handle WL configuration changes that affect the domain topology as it currently only 'introspects' configuration just before it starts the first server in a domain.  This means that on-line WLST/console changes that add or remove servers/clusters/network-channels won't be handles, and the operator won't know that it needs to support new pods/services for added resources or that it needs to delete services for removed resources.
You should not make dynamic changes to configuration that the operator is going to override for you.   E.g. your custom situational configuration, network addresses, log home location, etc.

Hope this helps!

Thanks for a very detailed explanation. This was very helpful.


feature works, now for GUI elements and such

addressed #31

can i refactor the loadGridFromFile and saveGridToFile methods to be somewhere else, like main.cpp? i feel like those methods are doing too much to be contained within GridController, for instance loading score and level data and soon to be loading and saving held pieces

access a variable contained within the main function and that seems wrong
What variable? @gillett-hernandez

The variable containing the held piece.

@gillett-hernandez I'm fine with whatever you decide.
Maybe the variable should not exist in main at all, and it should be a public variable on the GridController itself? considering the grid controller is what is controlling the pieces and where they are in the grid.


After setting marathon.url dcos-package-install will install onto that Marathon instance

After setting marathon.url in the dcos cli config (e.g. to access a marathon-user instance, compare https://github.com/mesosphere/dcos-cli/issues/396), the package command will use this as well. This is very confusing for the user.
In addition, services installed this way

will not show up in the DCOS UI
will not have a working admin router /service/<framework-name> endpoint.

marathon.url and core.dcos_url are effectively setting the same property. It should be one or the other. That said, I don't know why services won't show up in the DCOS UI when installed onto a marathon-user instance.

If you specifically set marathon.url I would think it confusing if we use it for only some requests to marathon but not all. I don't know why it doesn't show up in the UI, could you file a bug against the UI team please?


Refactored Screen code

Made a few static classes inside Screen to better organize it. E.g Screen.Fading is now use to check if the screen is fading etc

Why remove the spaces around operators and add a space after the cast operator?

@crosire that would be visual studio auto reformatting the casts

You can change that under Tools > Options > Text Editor > C# > Formatting.

@crosire but the space after castings option was turned off


DOC: boot strap 5 app

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 98.24%. Comparing base (8065ee1) to head (de15fa0).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #168   +/-   ##
=======================================
  Coverage   98.24%   98.24%
=======================================
  Files          33       33
  Lines         569      569
=======================================
  Hits          559      559
  Misses         10       10

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


hexfile: shrink memory image buffer

We're using a 50Mb memory buffer raw data that we're about to upload to the stm8 or that we've just downloaded from the stm8. If stm8gal runs on an embedded system with limited ram, this huge buffer is a problem.
The data in this buffer is transferred 1:1 to/from the stm8. It cannot be larger than the maximum flash size of the stm8, i.e. 256Kb.
Let's reduce the buffer size from 50Mb to 5Mb for now.

see https://github.com/gicking/stm8gal/pull/26


Authentication API MVP

Functioning authentication integration into webpage. Creation of Login Webservlet which has data fetched by JS and displayed on comments HTML.

Launched to: dterr-step-2020.uc.r.appspot.com


Vm instance creation

Fixed instance creation if deployment exists
check if the virtual machine deployment already exists. If so any new instance must be created as additional role to the deployment. Only the initial virtual machine instance must be created as a new deployment for the selected cloud service Fixes #144

Please test, I haven't found the time to extensively test this. Thanks

The code looks OK to me, I did not do any testing.
As a side note I think we need to look some more at the implementation philosophy as reviewing this code led me to believe there are cases when exceptions are used instead of return values. For example the test is a deployment exists or not (service.get_deployment_by_name) throws an exception, when it could simply return the deployment or None. But this is a bigger picture issue and should not influence this particular pull request.

agreed could be done nicer

The code nevertheless needs a real test before we merge, I won't have time to test today, thus if someone has a free slot earlier I would appreciate it

There are a number of real-world problems here, still. In addition to the existing code, we need to look at how we handle 'role-name', as this will affect how other config options (like creating data-disks) will be able to access the 2nd instance.

I was also concerned about the fact that deployment_name, role_name, label and service_name is always set to the cloud_service_name
on the first deployment this seems no problem, however if multiple instances run and role_name always equals cloud_service_name is it then possible to assign a data disk to the right instance ? We have this code here:
result = self.service.add_data_disk(
    cloud_service_name,
    instance_name,
    (role_name or cloud_service_name),
    lun,
    **args
)

so what is it based on ? I would say instance_name makes it clear, but am curious what role role_name and deployment_name plays here

What actual data maps back to role_name and deployment_name is not at all clear, and the only way to clarify will be through experimentation. We've made reasonable assumptions, but clearly they don't map to the reality :/
I can see now, though, that what we have doesn't work; we have two VMs in one of our infrastructure's cloud services, both of which were created through the portal (not azurectl) I cannot manage, for example, endpoints to the 2nd VM:
$ azurectl compute endpoint list --cloud-service-name=SMT1-Japan-West
{
  "endpoints": [
    {
      "idle-timeout": "4 minutes",
      "instance-port": "22",
      "name": "SSH",
      "port": "2222",
      "protocol": "tcp"
    }
  ]
}

$ azurectl compute endpoint list --cloud-service-name=SMT1-Japan-West --instance-name=SMT1-JP-West-i2
ERROR: AzureEndpointListError: 'AzureMissingResourceHttpError: Not Found\n<Error xmlns="http://schemas.microsoft.com/windowsazure" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">ResourceNotFound<Message>The deployment name \'SMT1-JP-West-i2\' does not exist.</Message></Error>'

I'm working through shell to try to figure out a good plan...

>>> svc = service.get_hosted_service_properties('SMT1-Japan-West', embed_detail=True)
>>> len(svc.deployments)
1
>>> [role.role_name for role in svc.deployments[0].role_list]
[u'SMT1-Japan-West', u'SMT1-JP-West-i2']
>>> [instance.instance_name for instance in svc.deployments[0].role_instance_list]
[u'SMT1-Japan-West', u'SMT1-JP-West-i2']

So, for instances created through the portal:

A cloud service is equivalent to a deployment.
Each virtual machine is created in it's own role

So it looks like we're consistent if we continue to use deployment_name = cloud_service_name , but we need to have role_name = instance_name.

I'll start a separate branch for fixes on instance modules that are affected, if you can just make sure we're creating instances consistently with the portal, here.

Instance modules are updated in https://github.com/SUSE/azurectl/pull/151

Thanks much, makes all sense to me. I followed your suggestion here too

I'll pull this down and test today. If practical testing looks good, I'll just merge it.

I'll pull this down and test today. If practical testing looks good,
I'll just merge it.

Thanks for doing this. I'll be off Thursday/Friday, feel free to just
work on the branch if we need additional changes

LGTM


Fixes ignored objects in one YAML file

Refs #5840
When a user runs kubectl create/delete -f file-with-multiple-resources.yaml and this file contains multiple objects separated with ---, kubectl ignores all objects except the first.
This PR reworked PathVisitor and DirectoryVisitor by merging them under the StreamVisitor. The idea is that directories, files and STDIN can all be handled as streams.
The problem until now was that only StreamVisitor was able to understand the --- and act accordingly, now with this merge we can be sure that in the future one change will reflect in all different was of input.

/cc @ghodss

@bgrant0607 is this v1 candidate?

any news on that?

I consider this a bug fix -- silently dropping the other objects is not a great user experience, so, yes, I'd like to fix this, or at least a clear error message.
However, I'm not really qualified to review it. Perhaps @smarterclayton or @deads2k could?

Or @ghodss?

This looks okay to me but it'd be great if @smarterclayton could take a quick look to ensure this consolidation is okay.

Please take a look on that. I refactored how files, directories and stdin are handled.
Every file must pass through the FileVisitor and then streamed through StreamVisitor. FileVisitor takes care of opening/closing files and StreamVisitor is parsing multiple resources.

@k8s-bot ok to test

@jlowdermilk PTAL

LGTM. Since this fixes a specific bug, should add a test (either unit or test-cmd.sh) to guard against regression.

@jlowdermilk I created a test-cmd.sh scenario but I am not really familiar with how those tests are working. Please review.

LGTM

LGTM, thanks for adding test.
Since you're unfamiliar, test-cmd.sh spins up a local kubernetes cluster and issues kubectl commands against it. Local cluster works well for testing client changes, as long as no provider-specific behavior is involved.

Thanks for the info. So is it ok the way I wrote it? Because I wasn't sure if it was ok to create a file on the same block with the test itself, here.

@bgrant0607 @davidopp for lead LGTMs

LGTM
Though I'd feel more comfortable if @jlowdermilk  could verify that the test is proper (see @andronat's last question).

@jlowdermilk thank you! done and rebased

lgtm again.


Add ViewManager.getAllExportedViews().

This method currently returns all registered cumulative views.  We could add a
way to specify which views should be exported later.  The API is currently
needed for the StackDriver exporter.
The NoopViewManager also returns all cumulative views.  However, the exporter
could choose to not export data when stats collection isn't enabled.

Coverage increased (+0.1%) to 81.209% when pulling 0d3d69dc1e6efc3ceab2ecba24b2c34825a461e9 on sebright:get-all-exported-views into 6fe9afd81e65230806d0f0d13bf75b233893beb8 on census-instrumentation:master.

Codecov Report

Merging #794 into master will increase coverage by 0.13%.
The diff coverage is 93.75%.

@@             Coverage Diff              @@
##             master     #794      +/-   ##
============================================
+ Coverage      79.1%   79.24%   +0.13%
- Complexity      890      895       +5
============================================
  Files           156      156
  Lines          3954     3970      +16
  Branches        373      377       +4
============================================
+ Hits           3128     3146      +18
+ Misses          747      746       -1
+ Partials         79       78       -1

Impacted Files
Coverage Δ
Complexity Δ

...src/main/java/io/opencensus/stats/ViewManager.java
100% <ø> (ø)
1 <0> (ø)
:arrow_down:

...ava/io/opencensus/implcore/stats/StatsManager.java
100% <100%> (ø)
8 <1> (+1)
:arrow_up:

.../io/opencensus/implcore/stats/ViewManagerImpl.java
100% <100%> (ø)
8 <3> (+3)
:arrow_up:

...io/opencensus/implcore/stats/MeasureToViewMap.java
97.18% <100%> (+0.04%)
23 <1> (+1)
:arrow_up:

...i/src/main/java/io/opencensus/stats/NoopStats.java
93.1% <87.5%> (-0.9%)
4 <0> (ø)

...census/implcore/trace/export/SpanExporterImpl.java
98.33% <0%> (+1.66%)
7% <0%> (ø)
:arrow_down:

...s/stackdriver/StackdriverExporterWorkerThread.java
87.5% <0%> (+5%)
10% <0%> (ø)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6fe9afd...0d3d69d. Read the comment docs.

Coverage increased (+0.09%) to 81.197% when pulling 19bed17801ddc67d1306ff7eb0e1dca0070d1436 on sebright:get-all-exported-views into f8e33e302547d209290611d00ea17b922c225375 on census-instrumentation:master.

Coverage increased (+0.08%) to 81.252% when pulling 1ae178ad816cf25a334ef717e37be05219dc9baa on sebright:get-all-exported-views into 9b50fa085c73ada5452e286cb030a33252d96cb4 on census-instrumentation:master.

Coverage increased (+0.08%) to 81.25% when pulling 548501b4ef03f54feb91453d150ef447277cd275 on sebright:get-all-exported-views into 9b50fa085c73ada5452e286cb030a33252d96cb4 on census-instrumentation:master.

Coverage increased (+0.08%) to 81.25% when pulling 5a2ad707e6a046a4035727ce0d20321839294b03 on sebright:get-all-exported-views into 9b50fa085c73ada5452e286cb030a33252d96cb4 on census-instrumentation:master.

Thanks!  I rebased and merged.


Transfer HSE prefabs from old project

Everything from the HSE room @MarkusJohansen

Everything from the HSE room @MarkusJohansen

Please describe what objects/prefabs in the description of the issue


ng lint fails on src/typings

Please provide us with the following information:

OS?  Windows 7, 8 or 10. Linux (which distribution). Mac OSX (Yosemite? El Capitan?)

OSX 10.11.6

Versions. Please run ng --version. If there's nothing outputted, please run
in a Terminal: node --version and paste the result here:

angular-cli: 1.0.0-beta.14
node: 6.5.0
os: darwin x64

Repro steps. Was this an app that wasn't created using the CLI? What change did you
do on your code? etc.

Add jQuery to typings.json (and run typings install to create src/typings/globals/jquery/index.d.ts)

The log given by the failure. Normally this include a stack trace and some
more information.

src/typings/globals/jquery/index.d.ts[19, 5]: Declaration of public instance member variable not allowed to appear after declaration of public instance member function
Among many other errors

Mention any other details that might be useful.

It would be nice to have a way to exclude folders ourselves (as of right now, ignoring in TSLint only works from a command line flag), or just automatically exclude src/typings be default

Thanks! We'll be in touch soon.

We use TSLint for linting, and basically just call npm run lint when you do ng lint.
You can edit the lint script in package.json to exclude that directory:
"lint": "tslint \"src/**/*.ts\" -e \"src/typings/**/*\"",

I'm running into an issue that seems counter to that last piece of advice, @filipesilva . Maybe you can help me out?
I'm trying to exclude a folder called /mocks/ when TSLint runs prior to my tests. I've added the following line to my scripts in package.json:
"lint": "tslint \"src/**/*.ts\" -e \"**/mocks/*.ts\"",
When I run npm run lint this faithfully ignores the /mocks/ folder, but when I run ng test, the folder gets linted.  Is it still the case that configuring the lint task in package.json should affect the lint executed prior to tests run for ng test?


motivate and explain parallel reduction, plus wordsmithing

This PR holds a small addition to the Confluence chapter.
It explains a bit more what can go wrong with the diamond
property and how that scenario motivates the definition of
parallel reduction.

Jeremy is the primary author of Confluence, so he should review.


fix ratio when set to “container”

setting ratio to container has broken since the ratio alogrithm was fixed.
When set to container it would use the old ratio algorithim to set the ratio.
This PR changes it so that it uses the correct algorithim based on wether or not traditionalRatio is set

Thanks!

Thank you

@aFarkas any timeline on publishing this PR?

give me a week

sorry it took longer.


[FIX-#399] Support TransactionSlot interface for reclaim-eth

Codecov Report

Merging #413 into master will increase coverage by 0.09%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #413      +/-   ##
==========================================
+ Coverage   60.64%   60.74%   +0.09%
==========================================
  Files          66       66
  Lines        3232     3235       +3
==========================================
+ Hits         1960     1965       +5
+ Misses       1272     1270       -2

Impacted Files
Coverage Δ

scenario_player/main.py
65.8% <100%> (+1.21%)
:arrow_up:

scenario_player/utils/legacy.py
30.52% <100%> (+0.49%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 420ae1b...f13e943. Read the comment docs.


New Items, Fixes and Rainbow Names

Added Rainbow Names
Added new items
Added parents and talismans in misc
Fixed some enchants and other Items

wait the vanilla enchantment table gui?

wait the vanilla enchantment table gui?

Yes https://twitter.com/SokoBuilds/status/1400420998143516674


code: updated to NullSafety

Simplest jump to null safety - a couple ? of changes in dart and one in yaml

Are you willing to publish a library separately? If the author has not merged

Are you willing to publish a library separately? If the author has not merged

Done already in my fork.
It seems no sense to make duplicated library in pub. One can simply link to it via syntax like that:
  flutter_absolute_path:
    git:
      url: https://github.com/ValeriusGC/flutter_absolute_path.git


Auto approve dependabot prs

Auto approve dependabot prs

coverage: 80.626% (-0.08%) from 80.707%
when pulling 3ffa1cafca9baac18c090b558cfd7f8c81de6671 on dependabot/MartinForReal-patch-1
into df19a87bfe512991b9b1ea5d2324d5e05dda4afd on main.


Using the downloaded weights doesn't create the expected result on the test data

Hi. I'm loving the great documentation there is in this code. I was able to do everything as the steps provided
I'm attaching some of the outputs after doing
bash downloadWeights.sh
python ../inference.py --exp_name "sd_inf" --input_image_json 'data/SD/SD_Test/SD_TEST.json' --output_image_folder './sd_output' --model_weights_path '../weightsFolder/BKS.pt'

I was hoping someone could help me with the issue I have with these outcomes

for example, look at the third segmented line of picture 1, and the third and fourth line of picture number 3
I'm running this on Windows 10, python version = 3.9

Kindly do the inference now and feel free to close the issue if it is solved.


[Core] Cold writing

Search before asking

[X] I had searched in the issues and found no similar feature requirement.

Description
Support writing cold data whose timestamp is out of the current writing window.
The block controller tends to create cold blocks to collect these data instead of opening the closed blocks online. The scanner and fetcher of the series should look at cold blocks for handling querying tasks. We also need a periodical job to merge cold and regular blocks.
Use case
No response
Related issues
No response
Are you willing to submit a PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

please assigned it to me

please assigned it to me

Thank you for being intered, but this is a core feature, Gao hss owned this.
It is better yoi could talk to him, and begin with some more friendly work.

We have supported this feature.


Add Lipo.io to the README

Just released https://lipo.io/ and have it's API running.  It's a free service that's a drop-in replacement for Sharp (has the same API and requires no dependencies nor sharp installation on the machine).
Curious if we could add this to the README?  Everything is completely open source:
https://github.com/lipojs/lipo
https://github.com/lipojs/api.lipo.io
https://github.com/lipojs/lipo-koa
https://github.com/lipojs/lipo-express
The API uses @ladjs/api under the hood:
https://github.com/ladjs/api

Hello, this looks very interesting, especially to the (hopefully diminishing) numbers who for whatever reason are unable to install sharp. Perhaps there should be a "built with sharp" page/section in the docs?
Off the top of my head, a few thoughts:

Is there a plan to deal with versioning, e.g. how might breaking API changes be handled?
Could the dependency on the deasync native module cause similar problems to those seen with sharp?
Might this be suitable for "containerisation" so people can self-host?
Who is paying for the hosting costs of this service? Are there any uptime SLAs etc.?

The concept behind this is great, thank you. I'll close for now but would love to learn more about API versioning problems and how this might be able to help those unable to deal with native modules.