Styling of the clause block

Discussion 🗣

Currently, the transformation of the clause to OOXML looks like this.

As visible from the image, the clause transformation is not visually distinguishable from the others as in case of templatemark-dingus or contract-editor.
Any ideas on how to distinguish the clause contents present in a template are welcomed.
Context

Raised with relation to the PR #432

I think wrapping it in the content control is sufficient.

@K-Kumar-01 does #432 close this Issue, or is this separate?

@irmerk This is a bit different, there wasn't a firm discussion on styling so I went with the basic one there which Dan approved at the time since we were focused on conversion. This needs some discussion on how we want it to look.

Pinging @michael-grover on this.

@Michael-Grover
Apologies for the late reply.

What does "wrapping it in the content control" mean? Can a Microsoft Word user see what content is wrapped in the content control versus what is not?

Yes a user can see if text is wrapped in content control or not. A border appears around the content wrapped when hovered as shown above.

I'm not familiar with the styling constraints on a Microsoft Word plugin. What are some options we have for visually distinguishing a clause from the content around it?

Not too sure myself. From what I have worked upon, one can try few of the following:
coloring the text, highlighting the text, changing the font, changing the size.
Although these are not good ones as size changes need to be specific. If a user will change the size, then we won't be able to say that it will still lie in a clause. Same for the other things.

My understanding is that the inside of this red box I drew is the clause. Is that correct? What is the grey box, and why does it wrap content that is not part of the clause? As a user I would expect the clause to have something like the gray box wrapped around it.

The gray part is the template area.It was implemented by @algomaster99 in his GSoC period. Later, I, @dselman, and @algomaster99 decided to wrap only the clause. But on thinking deeply, there was a problem which I found. That was how to determine which templates are already inserted and bind things.
Explanation: Suppose a template contains 2 clauses. Then we need to wrap both clauses along with content controls.
Also, there shouldn't be more than one template of the same kind. This is only possible via an id if given to the template which we do as of now. If only the clause is wrapped, then some ids need to be given to each clause which is associated with the template and on reopening further processing so that content controls are properly matched.

@K-Kumar-01 I would suggest looking into popular Word plugins that also differentiate a block of text from the whole document, and see some ways they achieve this. I don't think we should necessarily use styling such as font and size to differentiate clause text, because the author of the document may want their contract to look visually consistent.


Multipack padding fix

For non-dolomite models, packing_max_batch_len was being calculated incorrectly. This calculates the necessary increase to make sure that average batch size is similar to the specified effective_batch_size.

@aldopareja @Maxusmusti I have a couple minor concerns with this merge process-wise:

It looks like the lint CI job was failing. Ideally nothing should merge with a failing job.
The commit history is not in an ideal state for including in the repo. This is OK while you're in development, but merging all of these "fixes" commits isn't ideal. In this case, I would suggest squashing this all down to a single commit before merging.

Hey @russellb those are fair concerns. In this case:

the only lint failure was for an erroneous check that was removed from the repo, all of the formatting / linting checks passed

the PR was squashed on merge, so it is represented as a single commit with a more proper name / message in our history

Hey @russellb those are fair concerns. In this case:

the only lint failure was for an erroneous check that was removed from the repo, all of the formatting / linting checks passed

ok, thanks.

the PR was squashed on merge, so it is represented as a single commit with a more proper name / message in our history

Ah, I didn't notice that. That helps in this case.
However, note that the org-wide policy is to use merge commits. That does put more responsibilities on maintainers to ensure there's an intentionally structured commit history in a PR before merging it.  See this doc for more:
https://github.com/instructlab/dev-docs/blob/main/docs/github-merge-strategy.md
so we should be turning off the other merge methods at some point


Problem installing scikit-monaco

Hello,
I have tried to install scikit-monaco in Python2.7 and it is reporting an issue.
I have tried:

pip install scikit-monaco:

Failed building wheel for scikit-monaco
LINK : fatal error LNK1181: cannot open input file 'm.lib'

easy_install

scikit_monaco:
LINK : fatal error LNK1181: cannot open input file 'm.lib'

I tried to install directly, this is what happened:

Unable to find vcvarsall.bat
I know that problem should be related to compiler, but another packages can install without problems.
Thank you,
Albert

I forgot to assign it to @pbugnion

I'll try and have a look at this, but, truth be told, this repository isn't really maintained at present. If you want to have a go at fixing this, I'm happy to accept pull requests.

Dear Pascal,
Thank you for the response.
I will appreciate if you could check the problem. I have tried without alter the code (just the installers) and it is not working.
I am in the final way of my PhD and I need to do some integrations!!!
Thank you,
Albert

Albert R. Ortiz Lasprilla
Graduate Research Assistant
Phone: (1) 803 - 800 25 29
Email<EMAIL_ADDRESS>Web: http://sdii.ce.sc.edu/user/103
Department of Civil and Environmental Engineering
University of South Carolina
Columbia, SC. USA

From: Pascal Bugnion<EMAIL_ADDRESS>Sent: Saturday, February 06, 2016 6:18 AM
To: scikit-monaco/scikit-monaco
Cc: ORTIZ LASPRILLA, ALBERT
Subject: Re: [scikit-monaco] Problem installing scikit-monaco (#10)
I'll try and have a look at this, but, truth be told, this repository isn't really maintained at present. If you want to have a go at fixing this, I'm happy to accept pull requests.

Reply to this email directly or view it on GitHubhttps://github.com/scikit-monaco/scikit-monaco/issues/10#issuecomment-180738924.

Hello @alrortiz,
I hope it is not too late to response. And I assume you were to use skmonaco on Windows.
To solve the installing problem:
delete all 'libraries=["m"]' option in config.add_extension
install from setup.py
as for the problem of vcvarsall.bat, I guess it's due to the version of your VS common tools and it might work if you just change the environment variable

@olefinic Thanks for looking into this. I don't have access to a Windows machine to test this out.
If you have time, could you paste the revised skmonaco/setup.py (or open a pull request)?

@pbugnion Then I think it would influence the gcc users if delete 'libraries = ['m']' in such a simple way...⊙﹏⊙
Yes, indeed, having tested on OS X, I found skmonaco works well without any modification. The little trick is only for machine using vc, I guess.

Thanks for this. It'll definitely be useful for people wanting to use this on Windows!


Target class [toastr] does not exist.

Illuminate \ Contracts \ Container \ BindingResolutionException
PHP 8.1.13
Laravel 10.1.0
Target class [toastr] does not exist. in cPanel but Localhost is working

Add these items
Yoeunes\Toastr\ToastrServiceProvider::class,
Flasher\Laravel\FlasherServiceProvider::class,

to providers
and
'Toastr' => Yoeunes\Toastr\Facades\Toastr::class,
'Flasher' => Flasher\Laravel\Facade\Flasher::class,

to aliases
in config/app.php manually


Article single view explodes when summary contains image tags

When for some reason your summary contains images (aka HTML img tags) the single article view (or detail view, whatever it is called) is messed up. The reason is that the img tags show up in the meta description tag and are actually displayed, despite the summary not even being used on that page. This can be avoided by manually setting a summary for articles starting with images but that is not nice.
I am not sure if this is more a Pelican core issue or a theme issue so I start here because Pelican has lots of issues to work on already. :)

Mh I guess this is a Pelican issue, since if no Summary is explicitly set, it takes the first x characters of the content...


Add rxrepl

This PR adds rxrepl, a windows tool similar to sed -i or perl -pi.
I'm honestly not sure about the license with this one. I couldn't find an online version, so i had to upload it to hastebin. Please advise.

Why so messy regex?? "Version ([\\d\\.]+)" Should be enough.

@MCOfficer A handy tool for regex crafting: https://regex101.com/ 😁


Optional flag not working for sub-schemas

In version 3.4.0 of the package, setting optional: true on an object property which has its own required sub-fields, will enforce the object's sub-fields as required, even if the entire object is missing (which should be allowed, since the object itself is marked as optional).
See example below:
const addressSchema = new Schema({
    city: String,
});

const schema = new Schema({
    name: String,
    homeAddress: addressSchema,
    billingAddress: {
        type: addressSchema,
        optional: true,
    },
});

const context = schema.newContext();
context.validate({
  // EMPTY OBJECT
});

console.error(context.validationErrors());

The console output will show these errors:
0:{name: 'name', type: 'required', value: undefined} // OK
1:{name: 'homeAddress', type: 'required', value: undefined} // OK
2:{name: 'homeAddress.city', type: 'required', value: undefined} // OK
3:{name: 'billingAddress.city', type: 'required', value: undefined} // THIS IS WRONG!!!

Expected behavior:
The last error should not happen, of course because the entire billingAddress object is set as optional: true, therefore it's internal properties should only be enforced if (and only if) the object exists in the first place.
This expected behavior is also described in the documentation: https://github.com/longshotlabs/simpl-schema#optional

Same here, had to downgrade to 3.2.0, it's a regression

Yeah, I am also facing the same issue. Are there any plans?

I came across this issue as well. And after some debugging I found it's a change (a regression in my opinion) between 3.2.0 and 3.3.0.
Here is a sandbox to play with code, where you can switch between simpl-schema versions. I shortened the case to this schema:
const addressSchema = new SimpleSchema({
  city: String,
  street: String
});
const personSchema = new SimpleSchema({
  name: String,
  address: { type: addressSchema, optional: true }
});

Here, a person's address should be optional, but if it appears, it must contain a city and address. An object such as { name: 'Joe' } should pass validation, and it does in 3.2.0, but it does not in 3.3 and 3.4.
We found the issue while doing a round of package updates; we will stick for 3.2.0 for now, but I would appreciate if anyone knows why this was changed -- or otherwise, if there is any way to mark as optional nested objects with mandatory fields (I think that's a good description of  what the issue is).

We are facing the same issue. Can the change be reversed?
@aldeed

it´s somewhere in the commit e8f91dcd5016b3998cbec425fa77f337b398fc96 for "feat: improved oneOf support" by @znewsham and @aldeed
but i can´t find the bug

it´s somewhere in the commit e8f91dc for "feat: improved oneOf support" by @znewsham and @aldeed but i can´t find the bug

It seems the def.optional reference will never be evaluated as the def reference is only assigned within the for loop.   Leading to always assigning an {} empty object even for optional definitions.

Thought I'd share another gotcha with version 3.2.0 and its dependency on mongo-object 3.0.1 with a 'length' field bug!  The below image shows the 'collection' variable value, with a length value and causing the 'while' loop to work away...

Kind of stuck!   This latest version with the nested 'optional' misbehavior and the older version with its looping 'length' bug... lots of fun.

I'll open an issue on the mongo-object project for the each functions unexpected looping behavior when processing an object with length property but not really an array.  This package's clean function is using mongo-object's removeArrayItems function which uses the mentioned each utility function.
Something like this isArrayLike function may solve it, but I'll need to write a bunch of tests first to contemplate the scenarios thoroughly to ensure the addition of the object has numeric keys condition is desirable.
/**
 * Checks if the given object is array-like, meaning it is not null or undefined, has a finite length, and has numeric keys.
 * @param {Object} obj - The object to check.
 * @returns {boolean} - True if the object is array-like, false otherwise.
 */
function isArrayLike(obj) {
  return obj != null && typeof obj === 'object' && isFinite(obj.length) && obj.length >= 0 && obj.length === Math.floor(obj.length) && obj.length < Number.MAX_SAFE_INTEGER && (obj[0] !== undefined || obj[obj.length-1] !== undefined || Object.keys(obj).length == 0);
}

Looks like it's been over a year since this was reported -- any plans to fix this? To not fix this? Any update or suggestions for a temporary fix would be greatly appreciated. Thanks.

Thanks for your help debugging @moonrockfamily and others. I believe this should be fixed in 3.4.6


Tutorial Request

Is there a tutorial on how to actually add this into a script so the script can be whitelisted?
There is a script that I'm trying to add the ability for someone to use Discord Ace Perms with based on the roles in order to be able to activate the keybind command. But I'm having no luck. The script already comes with support for steam hex and such things but not ace perms. Thank you.
-- Standalone / Other
else
  Citizen.CreateThread(function()
    TriggerServerEvent(GetCurrentResourceName()..":CheckAccess")
  end)
end

hasAccess = false
RegisterNetEvent(GetCurrentResourceName()..":CheckAccess", function(bool)
  hasAccess = bool
end)

I dont think the guy who made the script knows how to do it. He set it up to allow esx and qbcore but for standalone he just put this... and said "its easy to setup" lol

I don't think this is a proper issue. This is moreso a request and this script is just for ACE permissions. Use FiveM's ACE permissions. No need to hook into this script.


[Extensibility Request] Page 364 "Select Sending Options" - Use ElectronicFormat

We would like to extend the Document Sending Profile but we want to use the code for setting the LastFormat.
On Page 364 "Select Sending Options", can we have the procedures public and the LastFormat and CustomTxt variable protected.
Variables to set protected:
protected var
        CustomTxt: Label 'Custom';
        LastFormat: Code[20];

Procedures to make public:
procedure SetSendMethodToCustom()

procedure LookupFormat(): Code[20]

procedure GetFormat(): Code[20]

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 10836.


esp-lwip weird ref 2e7efc4 (IDFGH-7608)

Environment

Development Kit:      [none]
Kit version (for WroverKit/PicoKit/DevKitC):
Module or chip used:
IDF version (run git describe --tags to find it):
v5.0-dev-3485-g20f5e180ee
Build System:         [idf.py]
Compiler version (run xtensa-esp32-elf-gcc --version to find it):
Operating System:     [Windows]
(Windows only) environment type: .
Using an IDE?: [No]
Power Supply:

Problem Description
Hi, I notice that the commit of the submodule 'lwip' seems like not on the espressif/esp-lwip repository.
But it can be checked out.
git.exe submodule update --progress --init --recursive --force -- "components/lwip/lwip"
Cloning into 'C:/Users/imcha/Desktop/esp-idf/components/lwip/lwip'...
remote: Enumerating objects: 50277, done.
remote: Counting objects: 100% (309/309), done.
remote: Compressing objects: 100% (139/139), done.
remote: Total 50277 (delta 192), reused 255 (delta 168), pack-reused 49968
Receiving objects: 100% (50277/50277), 10.07 MiB | 11.95 MiB/s, done.
Resolving deltas: 100% (37935/37935), done.
From https://github.com/espressif/esp-lwip
* branch              2e7efc486e6d90dfe8a6f058682375870c05071f -> FETCH_HEAD
Submodule path 'components/lwip/lwip': checked out '2e7efc486e6d90dfe8a6f058682375870c05071f'

Question:

Is this meet exceptation? I mean it works but the commit not 'exist' on the espressif/esp-lwip repo.
I wonder why it works OK. It would be great thanks if you can explain this to me.

Expected Behavior
Actual Behavior
Steps to reproduce
Code to reproduce this issue
Debug Logs
Other items if possible

Hi @imchangchang
Thank you for reporting this problem, I forgot to post the new lwip changes to a branch, as it currently contains some WIP commits.
The branch has just been pushed as 2.1.3-esp-temp

Is this meet exceptation? I mean it works but the commit not 'exist' on the espressif/esp-lwip repo.

We're still planing on updating the branch content, but this should still be an official branch in esp-lwip, so this is an issue, not expected.

I wonder why it works OK. It would be great thanks if you can explain this to me.

The commit is available in esp-lwip but didn't belong to any official branch. That's why it worked okay.

Thanks.

Thanks for reporting, feel free to reopen.


Curved text in custom complex shapes

This issue is related to https://github.com/jgraph/drawio/issues/1563 . However, a bit more detailed with additional real life use cases and current web implementations that would support this.
Curved text is often desired for many drawings and text that is located within a circle or ellipse. An example is an onion architecture

where the text wraps around the circle.
While I understand that there is currently no interest for implementing a shape/UI that offers the creation of this curved text, a great help would be to add support for textPath  in the custom shape editor, so we have the ability to keep using draw.io for these types of drawings (https://developer.mozilla.org/en-US/docs/Web/SVG/Element/textPath).
Currently the most supported method is to link a textPath to a path curve. However, for parsing - the newer "path" attribute can be used similar to the SVG 2 proposal: https://www.w3.org/TR/SVG2/text.html#TextPathElementPathAttribute
Comparing this issue to other drawing software: MS Visio does not even support this (https://support.microsoft.com/en-us/office/why-can-t-i-fit-a-line-of-text-to-a-curve-1081713f-c810-4bd4-b910-c409669d1c4b) with their solution being to use WordArt.

Declined

What was the reason for declining? Is there a similar method to achieve a curved text which is quite common in a lot of visualisations?


fix: re-fix for issue 23456

Description

Now JSON is rendered in ordered format

PR fixes following issue(s)
Fixes #23456
How Has This Been Tested?

[x] Manual
[x] JUnit

Summary by CodeRabbit

New Features

Enhanced JSON parsing to ensure the order of keys is preserved, improving data consistency.
Introduced support for new data types in the API plugin.

Refactor

Updated JSON handling logic for better performance and reliability.

Tests

Modified tests to align with the updated JSON parsing and header order.

/ok-to-test<EMAIL_ADDRESS>

/ok-to-test<EMAIL_ADDRESS>@tag.Datasource @tag.Fork, @tag.ImportExport @tag.Templates"

/ok-to-test<EMAIL_ADDRESS>@tag.Datasource @tag.Fork, @tag.ImportExport @tag.Templates"


how to determine if div already has embedded map?

Is there an easy way for me to determine whether or not a div already has a map embedded in it?
I have multiple views that require maps, so I constantly re-initialize the map on different divs. Is there a simple solution for this, or do I need to keep track of the active map div it with a service?

There is only one map, you cannot create multiple instances, it is an illusion. Checking if the map is initialized will initialize it. You could either hide it and show it, or call .remove() after each use.
' plugin.google.maps.Map.getMap(document.getElementById("map_canvas")).remove() '

Yep, there is only one map, but you can move it from div to div.
My question is whether of not there is a method for determining whether or not a div has an embedded map. There is a setDiv() method. Is there something like getDiv()?

only that .getMap method I used above, but like I said without the .remove() it's the same as initializing it originally.

I think the best solution is to be sure to clear your map once it's out of sight.


Generate a .hex file for STM32 boards

Improves build for STM32F769I

Out of curiosity whats the file size of the hex files?

Thanks for this :) Ill let the CI run then merge


boards: arm: add the Adafruit ItsyBitsy M4 Express

The Adafruit ItsyBitsy M4 express is a small (36 mm x 18 mm) ARM development board with an onboard RGB LED, USB port, 2 MiB of SPI flash, and range of I/O broken out onto 23 GPIO pins.

Thanks @MaureenHelm - I'm playing wtih https://micro-ros.github.io/ working on Zephyr, and ran out of RAM.  It'll be easier with the 192 KiB of RAM on this board :)

Done.  I also dropped an unneeded '0x' from the partition map.

Rebased post the 2.3 branch.


Improve Kotlin support documentation

Guice 5's release notes mentions:

Improved support for using Guice with Kotlin.

in https://github.com/google/guice/wiki/Guice501. However the wiki page still doesn't provide any example around how Guice 5 is better for kotlin compared to previous version.

It's described in the release notes:

Improved support for using Guice with Kotlin

Multibinder now binds Set<? extends T> so Kotlin code can inject
Set instead of Set<@JvmSuppressWildcards T>
MapBinder now binds Map<K, ? extends V> so Kotlin code can
inject Map<K,
V> instead of Map<K, @JvmSuppressWildcards T>

On Tue, Apr 20, 2021, 21:47 Vignesh @.***> wrote:

Guice 5's release notes mentions:
Improved support for using Guice with Kotlin.
in https://github.com/google/guice/wiki/Guice501. However the wiki page
https://github.com/google/guice/wiki/GuiceInKotlin still doesn't
provide any example around how Guice 5 is better for kotlin compared to
previous version.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google/guice/issues/1513, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAGMNHUCQ3HQAHL4RYG7CJ3TJXK6XANCNFSM43I56UEA
.

Is that it?

I helped myself by creating that class in my project. In case it helps anybody else, I added a gist for it over here: https://gist.github.com/lenalebt/d8d75e398c49dad256e4a0c005ab9a15
It probably is wrong somewhere, and probably also does not contain everything that would be necessary, but maybe it's a start for someone else to make a better version :smile:


[Fabric] Fix NativeComponentExample not rendering on Fabric

Please select one of the following

[ ] I am removing an existing difference between facebook/react-native and microsoft/react-native-macos :thumbsup:
[x] I am cherry-picking a change from Facebook's react-native into microsoft/react-native-macos :thumbsup:
[ ] I am making a fix / change for the macOS implementation of react-native
[ ] I am making a change required for Microsoft usage of react-native

Summary
The NativeComponentExample is currently not rendering when building for fabric on iOS.
ref._internalInstanceHandle?.stateNode?.canonical is undefined.
https://github.com/microsoft/react-native-macos/blob/main/packages/rn-tester/NativeComponentExample/js/MyNativeView.js#L38
Upstreamed the latest MyNativeView.js from https://github.com/facebook/react-native/blob/main/packages/rn-tester/NativeComponentExample/js/MyNativeView.js
Changelog
[macOS][Fabric] - Fix NativeComponentExample to render correctly
Test Plan
Before
https://user-images.githubusercontent.com/96719/213311648-2b5641cd-7754-4048-8808-2ee6f5f061e9.mp4
After
https://user-images.githubusercontent.com/96719/213311320-097131ab-1978-4376-951f-76fc20c70460.mp4

Thanks for the fix! When I had the code looking like this before, it failed the newly added Fabric CI. I'm curious to see if it will pass now :)

Oops, this broke Paper on iOS 😓

I'll fix it so that we only render when fabric is available

My understanding is that Turbomodules is somewhat broken at this point in time and the actual paper implementation for RNTMyNativeView isn't getting linked :/.
I'm hoping this will be resolved after we sync to 0.71

Found a temporary fix that won't throw error when trying to render on Paper.
Paper

Fabric


go generate pkg/render/applicationlayer

Description

pkg/render/applicationlayer modsecurity ruleset content was modified as part of required changes for https://github.com/tigera/operator/pull/2716
however, go generate was skipped by mistake and left the actual rendered config files outdated

this PR was done only by executing go generate ./pkg/render/applicationlayer

For PR author

[ ] Tests for change.
[ ] If changing pkg/apis/, run make gen-files
[ ] If changing versions, run make gen-versions

For PR reviewers
A note for code reviewers - all pull requests must have the following:

[ ] Milestone set according to targeted release.
[ ] Appropriate labels:

kind/bug if this is a bugfix.
kind/enhancement if this is a a new feature.
enterprise if this PR applies to Calico Enterprise only.

/merge-when-ready delete-branch

OK, I will merge the pull request when it's ready, leave the commits as is when I merge it, and delete the branch after I've merged it.

I'm sorry but I failed delete the branch after merging the pull request.


Add retries to FUSE calls

The FUSE client also needs to retry network calls, because if one fails, the kernel understands this as a permission denied and doesn't attempt a retry, leading to failed copies / writes.

The smarter way would be to incorporate retries directly in the client


Hook goblins up to the STS instances which have HasTrace instances

Background
For some of the STS rules, we have instances of HasTrace, which allow us to generate valid sequences of states and labels (signals) which transition between the states.
We want to apply the goblins genetic algorithm approach to mutating these traces, so that we can get interesting invalid sequences of labels (signals) which will trigger PredicateFailures.
Proposed Solution
Define
breedStsGoblins
  :: forall sts
   . (HasTrace sts, Goblin Bool (Signal sts))
  => PredicateFailure sts
  -> IO (Population Bool)

which will generate traces and then mutate them with goblins, using an objective function to reward the triggering of PredicateFailures.

I'm unable to update this, but it should belong to https://github.com/input-output-hk/cardano-ledger/milestone/19

Is this done @mhuesch ?

Yep! Sorry, let this go stale.

No problem! We have a lot to keep track of :D


Possible bug with port

https://github.com/juxt/edge/blob/master/src/edge/web_server.clj#L94
listener (yada/listener vhosts-model {:port port})]
Should that just be:
listener (yada/listener vhosts-model port)]
or
listener (yada/listener vhosts-model (:port listener))]
As is the line below?
 (infof "Started web-server on port %s" (:port listener))
Or have I missed something :)

As is typically the case - 30 seconds after opening this - it has become obvious to me that it is not a bug.


Redis Password Option

My actions before raising this issue

[x] Read/searched the docs
[x] Searched past issues

Expected Behaviour
Environment variables should be available to configure the Redis deploy with a password, and to incorporate said password into the RQ, Cacheops and RQ scheduler configurations in the cvat server.
Current Behaviour
No option to use passwords.
Possible Solution
RQ, Cacheops and RQ scheduler all have built-in options for setting password already, so the server would need to simply add them to its supervisord.conf and production settings.
Context
I'm trying to deploy an instance of CVAT in my company's kubernetes environment, and we require all our redis instances to have password components. Because there isn't even the option within the python code, we're going to need to custom-roll our solution by either forking the repo and running our own image builds, or mounting modified python settings files as configMaps. Neither of which is as clean as we'd like.
Your Environment

Using Docker images of version 1.3.0 for both server and UI

Next steps
You may join our Gitter channel for community support.

@ljades May I ask you to prepare a PR with your changes, or is this not possible for legal reasons or something else?

@azhavoro I can make a PR, sure!
Currently running into other issues, but once I know it works, PR it.
(Maybe this is deserving of another Issue, but the other issue I'm running into right now involves supervisord not working properly because almost all of the program workers keep erroring out with
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell

the runserver worker also errors with:
Failed to import module 'tf_detection_api_format.converter.py': Can't import tensorflow. Test process exit code: -9.

This only happens on kubernetes runs, not docker-compose runs, and I'm trying to figure out why.)

This was implemented in #5505.


Updating the quantum extension to add simulator support

This checklist is used to make sure that common guidelines for a pull request are followed.

For new extensions:

[N/A] My extension description/summary conforms to the Extension Summary Guidelines.

About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your PR is merged into master branch, a new PR will be created to update src/index.json automatically.
The precondition is to put your code inside this repo and upgrade the version in the PR but do not modify src/index.json.

quantum

/azp run

@bettinaheim Please add description for your PR and linter related issues if exist. And you should also add tests to cover your code.

@kairu-ms I edited the description for the PR. Regarding the tests, we are still in the process of integrating support for simulators to our service. The made edits are a prerequisite to allow us to proceed with that. We also have integration tests on our end.

@bettinaheim If you want to release, please update extension version and history. Thanks.


A list of videos supported by AMP

This addresses issue:

#41

I signed it!
2016-03-09 15:27 GMT+01:00 googlebot<EMAIL_ADDRESS>
Thanks for your pull request. It looks like this may be your first
contribution to a Google open source project. Before we can look at your
pull request, you'll need to sign a Contributor License Agreement (CLA).
[image: :memo:] Please visit https://cla.developers.google.com/
https://cla.developers.google.com/ to sign.
Once you've signed, please reply here (e.g. I signed it!) and we'll
verify. Thanks.

If you've already signed a CLA, it's possible we don't have your
GitHub username or you're using a different email address. Check your
existing CLA data https://cla.developers.google.com/clas and verify
that your email is set on your git commits
https://help.github.com/articles/setting-your-email-in-git/.
If you signed the CLA as a corporation, please let us know the
company's name.

—
Reply to this email directly or view it on GitHub
https://github.com/ampproject/amp-by-example/pull/69#issuecomment-194318741
.

--
Chiara Chiappini

Fantastic Chiara! Could you also add a sample for embedding a non-natively supported video platform via  amp-iframe? "You can always use amp-iframe to embed video from platforms without dedicated AMP components."
//cc @adewale

Two more things:

Can we find a better title, e.g. 'Integrating Videos in AMP: an Overview'
The title is formatted incorrectly. You don't need to add dashs to the name, this happens automatically. Make sure to use quotes when creating a new sample, e.g. gulp create --name 'Integrating Videos in AMP: an Overview' --dest src/30_Advanced.

PTAL

I updated #41 to reference videos without dedicated AMP components which are loaded inside an amp-iframe.

Nice!
Just a heads up that there are currently like 3 more of them in active pull requests.

PTAL.

We should use the correct upper and lowercase spelling of the video platforms (some companies are picky about this).
Question: does it make sense to sort the video platform alphabetically with amp-video at the beginning and amp-iframe as the last element?

If we want to order them alphabetically, then I would say amp-youtube comes first. Otherwise I don't understand the ordering you are suggesting and it's just confusing if a part of it is ordered and some not.

My thinking was:

amp-video first as it is the amp-native way
amp-iframe last as it is your last chance ;-)

But you're right - let's stay with simple alphabetical ordering.

Alphabetical a-z order. PTAL.

Yay! Little nit: amp-twitter < amp-video

ops! fixed.

ops! fixed.

LGTM


Fix copying worker addresses to clipboard

Fixes: https://github.com/prestosql/presto/issues/2865
Spans have invalid ids

Thanks for fixing it quickly. We need to include generated dist directory as https://github.com/prestosql/presto/pull/2495/files

@ebyhr I've noticed :)
presto-main/src/main/resources/webapp/dist/worker.js pushed


Update to latest CAPI release that merges webhook and reconcilers

We want to use latest CAPI release that contains the watch filter changes. That release also merges the webhook and reconcilers on the same Deployment.

Superseeded by https://github.com/giantswarm/cluster-api-app


Add changelog

It would be really helpful to estimate when to upgrade if there was a simple changelog somewhere. For example, I would now want to see what changes have been made between 3.0.0-beta10 and 3.0.3 - but I can't seem to find this info anywhere!

totally agree with @Req
Such a great resource and very hard to locate, should be captioned with Change Log on the main page.


Implement xor modifier.

I've implemented an xor modifier, which allows searches for every one byte xor modification to a string. Right now it is only implemented for text strings, but I'm debating on implementing it for hex strings too, though that may be significantly more difficult. Here's some examples of how it would work:
// This will match on "Uihr!qsnfs`l!b`oonu" which is "This program cannot" xor'ed with 0x01
rule a {
  strings:
    $a = "This program cannot" xor
  condition:
    $a
}

// This will match on "U\x01i\x01h\x01r\x01!\x01q\x01s\x01n\x01f\x01s\x01`\x01l\x01!\x01b\x01`\x01o\x01o\x01n\x01u\x01" which is "This program cannot" xor'ed with 0x01
rule b {
  strings:
    $a = "This program cannot" xor wide
  condition:
    $a
}

// Match on either ascii or wide single byte xor versions of "This program cannot".
rule c {
  strings:
    $a = "This program cannot" xor ascii wide
  condition:
    $a
}

// A case-insensitive single byte xor search. It generates all the strings first, then applies every xor to each.
rule d {
  strings:
    $a = "This program cannot" xor nocase
  condition:
    $a
}

Thanks! I’ll write up some docs soon and send those in!


How to tag an older commit in Git?

Example:
git tag -a v1.2 9fceb02 -m "Message here"
Where 9fceb02 is the beginning part of the commit id.
You can then push them up using git push --tags origin master
You can do git log to show all the commit id's in your current branch.
There is also a good chapter on tagging in the Pro Git book.

Just the Code
# Set the HEAD to the old commit that we want to tag
git checkout 9fceb02

# temporarily set the date to the date of the HEAD commit, and add the tag
GIT_COMMITTER_DATE="$(git show --format=%aD | head -1)" \
git tag -a v1.2 -m"v1.2"

# set HEAD back to whatever you want it to be
git checkout master


Opcode after jump executed on error

To do some basic tests with your PicoRV32 I wrote a testbench, connected a rom with a simple
bare metal program in it and and let it run.
I started with a tiny C-program that I compiled with the riscv-gnu-toolchain
#include <stdio.h>
void main(void)
{
  register int cpureg15 asm ("a5") = 0;
  //a5 = ABI-name of R15

  while (1)
  {
    cpureg15++;
  }
}

It compiled to :
  -- Test 1
00000000 <main>:
   0:   ff010113            addi    sp,sp,-16
   4:   00812623            sw  s0,12(sp)
   8:   01010413            addi    s0,sp,16
   c:   00000793            li  a5,0
  10:   00178793            addi    a5,a5,1
  14:   ffdff06f            j   10 <main+0x10>

The first thing I noticed was a xxxxxxxx on mem_addr[31:0] at about 150ns.
I NOP-ed the first three words since they were not required and the xxxxxxxx was gone.
  -- Test 2
00000000 <main>:
   0:   00000013            nop
   4:   00000013            nop
   8:   00000013            nop
   c:   00000793            li  a5,0
  10:   00178793            addi    a5,a5,1
  14:   ffdff06f            j   10 <main+0x10>

I decided to go after this effect later on.
The next thing I saw were the TRAP at 275ns, mem_addr=0x00000018 at 205ns and
mem_rdata =0x00000000 at 215ns.
As the riscv-spec points out on page 6 (Instruction length encoding) an opcode
of 0x00000000 shall lead to a trap. Since I indeed zeroed the unused portion
of the rom, I added a NOP at (the technically speaking unused) address 0x18 :
Now the TRAP was gone but register 15 stuck at zero and was obviously not incremented.
No wonder, since the two instructions that executed were at 0x14 and 0x18 (JUMP and NOP).
  -- Test 3
00000000 <main>:
   0:   00000013            nop
   4:   00000013            nop
   8:   00000013            nop
   c:   00000793            li  a5,0
  10:   00178793            addi    a5,a5,1
  14:   ffdff06f            j   10 <main+0x10>
  18:   00000013            nop

To ensure that the NOP at 0x18 was not only prefetched but also executed, I swapped the opcodes
of 0x10 and 0x18. Now R15 began incrementing.
  -- Test 4
00000000 <main>:
   0:   00000013            nop
   4:   00000013            nop
   8:   00000013            nop
   c:   00000793            li  a5,0
  10:   00000013            nop
  14:   ffdff06f            j   10 <main+0x10>
  18:   00178793            addi    a5,a5,1

I swapped the opcodes back and calculated a jump further back, aiming at 0x0C.
In fact 0x10, 0x14 and 0x18 were executed and R15 was incrementing.
  -- Test 5
00000000 <main>:
   0:   00000013            nop
   4:   00000013            nop
   8:   00000013            nop
   c:   00000793            li  a5,0
  10:   00178793            addi    a5,a5,1
  14:   ff9ff06f            j   0c <main+0x0c>
  18:   00000013            nop

I replaced the NOP at 0x18 with a second increment and in fact R15 got incremented twice per loop.
  -- Test 6
00000000 <main>:
   0:   00000013            nop
   4:   00000013            nop
   8:   00000013            nop
   c:   00000793            li  a5,0
  10:   00178793            addi    a5,a5,1
  14:   ff9ff06f            j   0c <main+0x0c>
  18:   00178793            addi    a5,a5,1

My guess is, that the offset of the jump instruction is added correctly to the program counter
but only after executing the opcode following the jump instruction by mistake.
My question is : can you reproduce this effect or is something wrong in my setup?
Sadly I'm no verilog guy, so there is no use in trying to debug your code on my own.
Vivado 2016.1, ARTY-board (Artix-7 XC7A35T), riscv32-unknown-elf-gcc (GCC) 5.3.0
Project:
picorv32test.zip

Your main problem is that your memory asserts mem_ready all the time, signaling to the picorv32 core that mem_rdata already contains the read data, when in fact you block ram adds one cycle delay. Either fix your mem_ready handling or simply do an async read in blockram7s.vhd:
--  memrw : process (clk)
--  begin
--    if rising_edge (clk) then
--      dout <= memory (to_integer(unsigned(addr)));
--    end if;
--  end process;

  dout <= memory (to_integer(unsigned(addr)));

Regarding the following instructions:
   0:   ff010113            addi    sp,sp,-16
   4:   00812623            sw  s0,12(sp)
   8:   01010413            addi    s0,sp,16

This is setting up a stack frame for main(). But because you are not using a libc (or any other startup code), your stack pointer is not set up (i.e. all 'x'). See the test firmware in firmware/ in the picorv32 source package if you need some example code. If you only need to set up a stack pointer, you can also consider using the STACKADDR module parameter to picorv32 to set the stack pointer on reset.

@wolf kicks ass =P thanks for maintaining all your awesome projects...
On Aug 27, 2016 8:15 PM, "Clifford Wolf"<EMAIL_ADDRESS>wrote:

Closed #13 https://github.com/cliffordwolf/picorv32/issues/13.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/cliffordwolf/picorv32/issues/13#event-769798349, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAtRr-30-TQufQT9U3OcQoRn3fo8U8Cyks5qkH7agaJpZM4JuvFm
.

Wow, that was fast! Thank you very much.


MacVim --clean consumes one argument

Describe the bug
MacVim (latest master, commit d552421a21b24493ed0745980cf0bb267b021289) consumes an argument and opens 1 less windows than files (or no files when one file supplied)
This doesn't happen in terminal Vim from the same build or when --clean is not supplied even with an empty configuration.
To Reproduce
Detailed steps to reproduce the behavior:

Build MacVim in directory /Users/eirnym/dev/macvim using git clean -fdx && make clean && ./configure --enable-python3interp=dynamic && make
Create test files with cd /Users/eirnym/test && rm -rf *txt && touch {1,2,3}.txt
Open Vim with command src/MacVim/build/Release/MacVim.app/Contents/MacOS/Vim --clean *txt
When Vim (terminal) opens, execute :ls to make sure that all 3 buffers on the list, exit Vim. (it's correct behaviour)
Open MacVim with command src/MacVim/build/Release/MacVim.app/Contents/MacOS/MacVim --clean *txt (or by listing them exactly)
Only 2 windows were opened instead of 3
When you specify only one file, no windows will be open

Expected behavior
A buffer with first file is opened
Environment (please complete the following information):

Vim version commit d552421a21b24493ed0745980cf0bb267b021289 (currently the latest)
OS: macOS 10.15.6 (19G60d)
Terminal: Terminal.app

Seems like this is a longstanding bug. Will need to take a look.
Either way, if you want to launch MacVim with --clean, it's probably better to use the bundled mvim script or use src/MacVim/build/Release/MacVim.app/Contents/MacOS/Vim -g --clean *txt.

Thank you @ychin
Why MacVim binary is not correct version to run? If it's broken somehow why it shouldn't be fixed?
mvim script is not useful in this case as I want to run freshly built app without touching installed. I don't need auto discovery for MacVim's location.
For general use I have my own alias pointing to /Applications/MacVim.app/Contents/MacOS/Vim -g

Ps: Vim -g indeed works correctly.
Should we focus on MacVim binary here? For me it's a simple wrapper for Vim executable so I expect it works the same

MacVim is the raw binary for the macOS app container and shouldn't need to handle command parsing. It's not the same binary as Vim, which is actually the Vim program. In general, the "standard" way to call macOS apps is to use open: open /Applications/MacVim.app. Otherwise, doing /Applications/MacVim.app/Contents/MacOS/Vim -g works.
Alternatively, you can get your familiar gvim, vim, view, etc from /Applications/MacVim.app/MacVim.app/Contents/bin/, so you can just call /Applications/MacVim.app/Contents/bin/gvim if you want. They all just symlink to mvim anyway which is what you probably want to use (whether you freshly install it or not should have no relevance because it works by looking at its own locaiton).


Slow HTTPS

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
go version go1.9.2 linux/amd64
Does this issue reproduce with the latest release?
Not tested.
What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/gfyrag/work/gospace"
GORACE=""
GOROOT="/usr/lib/go"
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build524283905=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
What did you do?
package test

import (
	"testing"
	"net/http"
	"net/http/httptest"
	"io"
	"io/ioutil"
)

var (
	data = make([]byte, 1024*1024*100)
	h = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		w.Write(data)
	})
)

func BenchmarkHTTP(b *testing.B) {
	srv := httptest.NewServer(h)
	defer srv.Close()

	b.ResetTimer()
	for i := 0 ; i < b.N ; i++ {
		res, err := srv.Client().Get(srv.URL)
		if err != nil {
			b.Error(err)
			continue
		}
		_, err = io.Copy(ioutil.Discard, res.Body)
		if err != nil {
			b.Error(err)
		}
	}
}

func BenchmarkHTTPS(b *testing.B) {
	srv := httptest.NewTLSServer(h)
	defer srv.Close()

	b.ResetTimer()
	for i := 0 ; i < b.N ; i++ {
		res, err := srv.Client().Get(srv.URL)
		if err != nil {
			b.Error(err)
			continue
		}

		_, err = io.Copy(ioutil.Discard, res.Body)
		if err != nil {
			b.Error(err)
		}
	}
}

What did you expect to see?
A small loss on https.
What did you see instead?
A big loss on https :
BenchmarkHTTP-8    	      30	  41739281 ns/op
BenchmarkHTTPS-8   	      10	 148254359 ns/op

I know https is expected to be slower than http. But this seems to be  really slow.

3.5x slowdown, considering you’ve removed all network latency, sounds reasonable to me.

This bug is also a bit too vague to be actionable.
It's also somewhat of a dup of more specific bugs #21525, #22809, #20058, #19840, #22807, #22806, #20967, and a dozen others.
I'm going to close this one, but feel free to file other specific issues.

@bradfitz Yes, this was more a question than a bug. I found some others issues like those you mentionned, but most of them talk about tls handshake, which, i think is a problem with many small https request. In this case, the requests fetch 100 MB of data each time. Some of them talk about arm64 architecture which is not relevant in this case.
@davecheney Thks for the answer. I tought the slowdown will be of few percent (many blog post on the net seems to show that), but i was wrong.

@davecheney Thks for the answer. I tought the slowdown will be of few percent (many blog post on the net seems to show that), but i was wrong.

once network latency is added, I expect the benchmarks would show a smaller difference.

Hi,
I have a very simple code which retrieve one metric from aws clouwatch with the aws go sdk :
svc := cloudwatch.New(session.New(), &aws.Config{Region: aws.String(region)})
        dur, _ := time.ParseDuration("300s")
        params := &cloudwatch.GetMetricStatisticsInput{
                StartTime:  aws.Time(time.Now().Add(-dur)), // Required
                EndTime:    aws.Time(time.Now()),           // Required
                MetricName: aws.String(metric_name),        // Required
                Namespace:  aws.String("AWS/ELB"),          // Required
                Period:     aws.Int64(300),                 // Required
                Statistics: []*string{ // Required
                        aws.String(metric_stat), // Required
                },
                Dimensions: []*cloudwatch.Dimension{
                        { // Required
                                Name:  aws.String("LoadBalancerName"), // Required
                                Value: aws.String(elb_name),           // Required
                        },
                },
                //Unit: aws.String("Count"),
        }
        resp, err := svc.GetMetricStatistics(params)

until now I compiled the program with go 1.8 and it works very fine :

time ./elb-check.go187 eu-west-1 test-elb Latency Average
{
Datapoints: [{
Average: 0.15102177378775059,
Timestamp: 2018-02-20 11:01:00 +0000 UTC,
Unit: "Seconds"
}],
Label: "Latency"
}
real    0m0.057s
user    0m0.028s
sys     0m0.008s

But now I upgraded my go package to 1.9, I noticed that time execution increased to more than 2x compared to 1.8 binary without any change in the code :

time ./elb-check.go19 eu-west-1 test-elb Latency Average
{
Datapoints: [{
Average: 0.15102177378775059,
Timestamp: 2018-02-20 11:01:00 +0000 UTC,
Unit: "Seconds"
}],
Label: "Latency"
}
real    0m0.123s
user    0m0.104s
sys     0m0.000s

the cpu consumption is also higher, I suspect the overhead comes from the SSL support in go 1.9 performed to cloudwatch to get the elb metric in my case.
Here is the pdf from pprof for the new problematic version :
out.pdf

@xp-1000, this issue is closed and thus is not tracked.

my bad @bradfitz I thought it will be better to add information to existing issue and reopen it instead of open a new one.


Calculating NDCG@K of Different Groups

Is it possible to calculate the difference of NDCG@K between different groups given in the sensitive attribute?
IE: NDCG@K for group1(males) - NDCG@K for group2(females)

I get you! Of couse, you just calculate specific-group metric here
https://github.com/TangJiakai/RecBole-FairRec/blob/8c4ef324abddc4e9435f9c18aab9e0ba2d36386f/recbole/evaluator/base_metric.py#L67-L82
you just need pass the sensitive attribute of users and calculate group' metrics separately, you could modify this by yourself. And In future, I will consider adding this important function! (Welcome to PR!)

Is it possible to calculate the difference of NDCG@K between different groups given in the sensitive attribute? IE: NDCG@K for group1(males) - NDCG@K for group2(females)

Bro were you finally able to get that to work? i could not. thanks.

@Keyvantajfar
Yeah I was in a rush and couldn't figure out how to access the sensitive attribute values vector so I imported it and split the data by this attribute, then calculated the metric on each dataset. It's not good practice but it worked.
class NDCG_sep(TopkMetric):
def __init__(self, config):
    super().__init__(config)
    self.sst_attr_list = config['sst_attr_list']

def calculate_metric(self, dataobject):
    sst_value = self.sst_attr_list[0]
    #sst = dataobject.get('data.' + sst_value).numpy()
    #unique_value = np.unique(sst)

    test_df = pd.read_csv('./dataset/geo_data/geo_data.user', sep = '\t')
    sst_column = np.array(test_df[sst_value + ':float'])
    unique_value = np.unique(sst_column)

    metric_dict = {}
    for value in unique_value:
        pos_index, pos_len = self.used_info(dataobject)
        pos_index = pos_index[sst_column==value]
        pos_len = pos_len[sst_column==value]
        result = self.metric_info(pos_index, pos_len)
        metric_dict.update(self.topk_result(f'ndcg_class_{value}', result))

    return metric_dict

def metric_info(self, pos_index, pos_len):
    len_rank = np.full_like(pos_len, pos_index.shape[1])
    idcg_len = np.where(pos_len > len_rank, len_rank, pos_len)

    iranks = np.zeros_like(pos_index, dtype=np.float)
    iranks[:, :] = np.arange(1, pos_index.shape[1] + 1)
    idcg = np.cumsum(1.0 / np.log2(iranks + 1), axis=1)
    for row, idx in enumerate(idcg_len):
        idcg[row, idx:] = idcg[row, idx - 1]

    ranks = np.zeros_like(pos_index, dtype=np.float)
    ranks[:, :] = np.arange(1, pos_index.shape[1] + 1)
    dcg = 1.0 / np.log2(ranks + 1)
    dcg = np.cumsum(np.where(pos_index, dcg, 0), axis=1)

    result = dcg / idcg
    return result

@Keyvantajfar Yeah I was in a rush and couldn't figure out how to access the sensitive attribute values vector so I imported it and split the data by this attribute, then calculated the metric on each dataset. It's not good practice but it worked.
`class NDCG_sep(TopkMetric):
def __init__(self, config):
    super().__init__(config)
    self.sst_attr_list = config['sst_attr_list']

def calculate_metric(self, dataobject):
    sst_value = self.sst_attr_list[0]
    #sst = dataobject.get('data.' + sst_value).numpy()
    #unique_value = np.unique(sst)

    test_df = pd.read_csv('./dataset/geo_data/geo_data.user', sep = '\t')
    sst_column = np.array(test_df[sst_value + ':float'])
    unique_value = np.unique(sst_column)

    metric_dict = {}
    for value in unique_value:
        pos_index, pos_len = self.used_info(dataobject)
        pos_index = pos_index[sst_column==value]
        pos_len = pos_len[sst_column==value]
        result = self.metric_info(pos_index, pos_len)
        metric_dict.update(self.topk_result(f'ndcg_class_{value}', result))

    return metric_dict

def metric_info(self, pos_index, pos_len):
    len_rank = np.full_like(pos_len, pos_index.shape[1])
    idcg_len = np.where(pos_len > len_rank, len_rank, pos_len)

    iranks = np.zeros_like(pos_index, dtype=np.float)
    iranks[:, :] = np.arange(1, pos_index.shape[1] + 1)
    idcg = np.cumsum(1.0 / np.log2(iranks + 1), axis=1)
    for row, idx in enumerate(idcg_len):
        idcg[row, idx:] = idcg[row, idx - 1]

    ranks = np.zeros_like(pos_index, dtype=np.float)
    ranks[:, :] = np.arange(1, pos_index.shape[1] + 1)
    dcg = 1.0 / np.log2(ranks + 1)
    dcg = np.cumsum(np.where(pos_index, dcg, 0), axis=1)

    result = dcg / idcg
    return result`

Thanks for your help. Appreciate it.

@Keyvantajfar Yeah I was in a rush and couldn't figure out how to access the sensitive attribute values vector so I imported it and split the data by this attribute, then calculated the metric on each dataset. It's not good practice but it worked.
thanks again, but i have some issue with this code. how can i read sensitive attributes (or all columns) of the recommendation dataset. i mean how can i retrive the recommendation subset dataset which is the iterations users for example and then i select sensitive attributes of them with a WHERE id = id from the users info dataset. what you did here just works with recbole which i dont know why but it doesnt work with other datasets due to the issue i addressed. the len of pos_index of for example 1376 wont match the len of SST_column which is the same as all the users info i have but the model only chooses 1376 interactions for its recommendation so it wont work. thanks. if you could give me your email address i'd be much happier.

Like I said, I imported the specific dataset I used (geo_data) to access the sensitive attributes. This code will not work unless you change it, and therefore it is not good practice. Please contact the GitHub authors if you need further help.

I get you! Of couse, you just calculate specific-group metric here
https://github.com/TangJiakai/RecBole-FairRec/blob/8c4ef324abddc4e9435f9c18aab9e0ba2d36386f/recbole/evaluator/base_metric.py#L67-L82
@TangJiakai
you just need pass the sensitive attribute of users and calculate group' metrics separately, you could modify this by yourself. And In future, I will consider adding this important function! (Welcome to PR!)
Hello. i have a question about this if you'd let me have your email address i'd be much happier. i have scraped all the documents on the recbole but werent successful to find how the data part i have in my mind works. thanks in advance.

@bhuebner3
Like I said, I imported the specific dataset I used (geo_data) to access the sensitive attributes. This code will not work unless you change it, and therefore it is not good practice. Please contact the GitHub authors if you need further help.

thanks for your reply.


CLI initial version

NOTE: Further changes to this branch should be done by a PR

from the work we did on https://github.com/carletex/create-dapp-example (real-se2-templates / feat/extension-templateConifg-dgrcode branches)

This PR changes the workflow for users to use scaffold-eth-2. Instead of cloning, users will use this CLI (npx create-whatever-name@latest). It'll let users pick different options (hardhat/foundry + extensions)

Initially, we'll start with hardhat/foundry (so we can merge and iterate), but @dgrcode built an extension system. Read more here
Some ToDos (maybe we should create some issues?):

[ ] Update README (add new way of installing it, dev notes for the cli, etc)
[ ] GH Action to auto-publish to npm (extra: auto-increment version on release/merge)
[ ] Check node version (=> 18) and yarn installed before running. We could also have a "dependencies" file (or in config.json) for each extension, so we check them before installing (e.g. foundry extension needs to have foundryup in the system)
[x] Better ASCII art (or just chalk with color and fancy chars) on the intro message. (render-intro-message.ts) Fixed https://github.com/scaffold-eth/scaffold-eth-2/pull/413/commits/42e26bb18c8247b614ac6cd5a4b22433cfad6828
[x] In the outro message hint about yarn chain / yarn deploy / yarn start (render-outro-message.ts). Fixed https://github.com/scaffold-eth/scaffold-eth-2/pull/413/commits/da4395063eb23785695a5b778a674c0063961165
[x] Fix foundry template: Let's remove the need of having to create a .env file manually (the cli can do it for you... for now, until we find a system for extensions to add their own .env) Fixed https://github.com/scaffold-eth/scaffold-eth-2/pull/413/commits/8e05bbbe92568c4e6b41dd17fe29a027053eb357
[ ] Create the graph extension
[ ] Submodules: Foundry uses submodules, and we are setting that up directly on the CLI. It'd be ideal to move this to the template (e.g. the template declares some submodules, and the CLI dynamically set them up for every extesion)
[ ] Contributions: Find the best way to test PRs, develop new features with this system without having to copy/paste from a "installed instance". Something like hot-reloading
[ ] Contributions: Move to a release branch model (TLDR: instead of merging directly to main, use relase branches: eg. release-2.1.0 to batch stuff and the merge to master. Only hotfix goes directly to main)
[ ] Add prettier / husky / linting (and maybe remove some of them from the base template)

We don't need to solve all of them before merging.
PS: There are a lot of "files changed", but most of them are just moving things to the template folders (which git is detecting :))

Went for create-eth which was suggested by @austintgriffith and it was free :D
npx create-eth@latest
Could you guys try? @sverps @rin-st @Pabl0cks @damianmarti  (not tagging @technophile-04 because he is next to me IRL rn :D)

@carletex this is awesome!! :-)
I tested it trying to think that I'm a new user of SE-2 and I think that these improvements can be good things to do:

Add a progress bar when installing the dependencies with yarn (it takes some time and I think that the loading is not enough, at least for me I like to see what is happening or at least a progress bar moving). I don't know how easy or hard is to add it, I like to use the command pv on Ubuntu for this kind of thing...
It's on the ToDo list, but I think that after finishing we can add a short version of the README file, and add a reference to where the user can find more information (full README file or whatever).
Add some information about installing extensions and the available extensions.

I used the installed se-2 project and everything seems fine.
We are close to having the cli ready!! :partying_face:

Hey, great job!
For now I'm getting an error when trying to preinstall packages for both hh and foundry (Install packages? y). Part of the log:
...
➤ YN0007: │ bufferutil@npm:4.0.7 must be built because it never has been before or the last one failed
➤ YN0007: │ utf-8-validate@npm:5.0.10 must be built because it never has been before or the last one failed
➤ YN0009: │ se-2@workspace:. couldn't be built successfully (exit code 1, logs can be found here: /private/var/folders/rs/t2dhts8n6hv59mn9srqyh74r0000gn/T/xfs-b42708b1/build.log)
➤ YN0009: │ deasync@npm:0.1.28 couldn't be built successfully (exit code 1, logs can be found here: /private/var/folders/rs/t2dhts8n6hv59mn9srqyh74r0000gn/T/xfs-dd82bdf5/build.log)
➤ YN0007: │ ssh2@npm:1.14.0 must be built because it never has been before or the last one failed
➤ YN0000: └ Completed in 18s 70ms
➤ YN0000: Failed with errors in 35s 91ms

    at makeError (/Users/liana/Documents/se-2/node_modules/pkg-install/node_modules/execa/index.js:174:9)
    at /Users/liana/Documents/se-2/node_modules/pkg-install/node_modules/execa/index.js:278:16
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  code: 1,
  stdout: '➤ YN0000: ┌ Resolution step\n' +
    '➤ YN0032: │ keccak@npm:3.0.3: Implicit dependencies on node-gyp are discouraged\n' +
    '➤ YN0061: │ debug@npm:3.2.6 is deprecated: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)\n' +
...

build.log files:
# This file contains the result of Yarn building a package (se-2@workspace:.)
# Script name: postinstall

husky - .git can't be found (see https://typicode.github.io/husky/#/?id=custom-directory)

and
# This file contains the result of Yarn building a package (deasync@npm:0.1.28)
# Script name: install

Usage Error: Couldn't find a script name "node-gyp" in the top-level (used by deasync@npm:0.1.28). This typically happens because some package depends on "node-gyp" to build itself, but didn't list it in their dependencies. To fix that, please run "yarn add node-gyp" into your top-level workspace. You also can open an issue on the repository of the specified package to suggest them to use an optional peer dependency.

$ yarn run [--inspect] [--inspect-brk] [-T,--top-level] [-B,--binaries-only] <scriptName> ...
Build failed

so something wrong with husky, .git or node-gyp. Tried to install node-gyp and change node version to 18 but it didn't help. Will try to debug and understand how it works later today, but not sure it will be fast.
Other than that noticed that foundry project readme should be changed, but it's already mentioned. And name of the local network for foundry shouldn't be hardhat :)

@rin-st did you get this when doing it using npx create-eth@latest or you were trying it out locally using yarn cli ?
Because we were getting this error when you are trying it out locally using yarn cli and you select the project name in the current directory (where all the cli stuff is), like you need to create it outside this project directory :
eg :
> yarn cli

 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 | Create Scaffold-ETH 2 app |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+

?  Your project name: ../my-dapp-example

We need to document this problem in README while working locall....But could you please confirm if you got this error while doing npx create-eth@latest or doing yarn cli locally ?

@technophile-04 I have an error in both npx create-eth@latest and yarn cli. But using ../my-dapp-example works for both cases 👍

btw I was trying to use npx create-eth@latest from se-2 folder. If I use it from outside git repo it works as expected

Awesome job! Was exciting to see @carletex and @technophile-04 working on this IRL 🚀
On Windows (both my desk and laptop) it's working like a charm when I choose hardhat, but when I choose foundry, I got problems to yarn chain.
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 | Create Scaffold-ETH 2 app |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+

? Your project name: test3
? What solidity framework do you want to use? foundry
? Install packages? Yes

  √ 📁 Create project directory C:\Users\Pablo\test3
  √ 🚀 Creating a new Scaffold-ETH 2 app in test3
  √ 📦 Installing dependencies with yarn, this could take a while
  √ 📡 Initializing git repository

  Congratulations! Your project has been scaffolded! 🎉

  Next steps:

  cd test3

        Start the local development node
        yarn chain

        In a new terminal window, deploy your contracts
        yarn deploy

        In a new terminal window, start the frontend
        yarn start

  Thanks for using Scaffold-ETH 2 🙏, Happy Building!

$ cd test3

 ~/test3 (main)

$ yarn chain
"anvil" no se reconoce como un comando interno o externo,
programa o archivo por lotes ejecutable.
command not found: anvil

Thanks for the feedback and the help @rin-st.

btw I was trying to use npx create-eth@latest from se-2 folder. If I use it from outside git repo it works as expected

Yep, this was happening when running it inside an existing repo command (because of the postinstall yarn hook). Your fix works perfectly, but it leaves the yarn.lock file uncommitted (changed after running yarn install**), so we decided to take a different approach (git init before yarn install, but add & commit after yarn install) and it seems to work. @technophile-04 will push in a bit.
** This is expected since we have an empty yarn.lock file, to avoid conflicts when we have a bunch of extensions. So it's generated on the first install. In package.json files, we changed ^ for ~, so with packages bugfixes there is less risk of something breaking (we tried to do exact versions in package.json, but for some reason the peer-dependencies weren't installing)

On Windows (both my desk and laptop) it's working like a charm when I choose hardhat, but when I choose foundry, I got problems to yarn chain.

Thanks @Pabl0cks ! Yes, this happens because people need to install foundryup (that contains anvil and some other tools) in their system. What we are going to do is check if they have it installed and if not show a message hinting them to install.
We'll be pushing (& publishing) in a bit.
Thanks all!

Thanks @Pabl0cks ! Yes, this happens because people need to install foundryup (that contains anvil and some other tools) in > their system. What we are going to do is check if they have it installed and if not show a message hinting them to install.

Thx @carletex I had no idea about that. Now with foundryup is working great!
Getting the informative message before installing it, so I know I have to install foundryup => Installed it and tried again, and  CLI is working perfect with foundry also, got my working SE-2 🙌
I'll send a nitpick edge case:

When selecting foundry, and selecting "No" at the "Install packages?" option, it shows a finish icon at the Git step, but I don't get any feedback or next steps after instalation. With hardhat and "No packages" I get the congrats message with the steps to start SE-2.

That's all I get with foundry:

$ npx create-eth@latest
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Create Scaffold-ETH 2 app |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
? Your project name: test9
? What solidity framework do you want to use? foundry
? Install packages? No
√ 📁 Create project directory C:\Users\Pablo\test9
√ 🚀 Creating a new Scaffold-ETH 2 app in test9
↓ 📦 Installing dependencies with yarn, this could take a while [skipped]
→ Manually skipped
√ 📡 Initializing Git repository

When selecting foundry, and selecting "No" at the "Install packages?" option, it shows a finish icon at the Git step, but I don't get any feedback or next steps after instalation. With hardhat and "No packages" I get the congrats message with the steps to start SE-2.

Yes @Pabl0cks! I think Shiv solved it here: https://github.com/scaffold-eth/scaffold-eth-2/pull/413/commits/9bb9422b072fa52fa233214acd5c11516647618e
I haven't published it yet to NPM but it should work if run the CLI locally (instead of create-eth)

@dgrcode

Shouldn't we commit the dist/ folder? I'm curious how npx is working without the dist folder. Do they run the build task on their servers?

Yep, it builds there

What do you think about committing the .vscode/ folder to keep the debugger configuration? It definitely helps me, and I guess it could help anybody trying to debug the cli.

We decided against it in SE-2. Not everyone uses VSCode (eg. I use Intellij and Shiv uses Vim). The same way that I don't want to commit the .idea folder from my Intelij
The ToDos look great and pretty straightforward. Also remember to take a look at the "how to contribute" stuff (hot reloading, symlinks, different repo) which is the most important piece.

Also remember to take a look at the "how to contribute" stuff (hot reloading, symlinks, different repo) which is the most important piece

Yes, will do! but it could probably go in a different PR, no?

@dgrcode

Yes, will do! but it could probably go in a different PR, no?

Yep!
As we also talked offline, I'm going to protect this branch in the repo settings, so people will need to make a PR before merging here.
Thanks Dani!

opened https://github.com/scaffold-eth/scaffold-eth-2/pull/417 for the bug above

Bug : It seems that yarn next:types after fresh install of any options. :

To reproduce :

npx create-eth@latest
Choose none or any tempate
Yes for yarn install (you can cd and then do yarn install)
Run : yarn next:check-types

I just opened the PR https://github.com/scaffold-eth/scaffold-eth-2/pull/423 to remove an item from my TODO list.

Wow, nice. Just ran npx create-eth@latest on my Mac Book Air M2, MacOS version 13.4.1 (22F82), Node version v18.15.0 and Yarn version 3.2.3 and it was smooth. I choose Foundry to check this out and it worked well. I really liked that it checked if Foundry was installed and told me how to install it: (NOTE: Foundryup is not installed in your system). That is great developer experience 🤩.
Two small remarks:

I personally don't like submodules. They can get easily messy quickly. Especially junior devs that are not familiar with it can quickly run into weird problems. But I assume that comes upstream from Foundry. Isn't there a way to run/install forge and anvill without submodules?
packages/nextjs/generated/deployedContracts.ts is marked as changed. It seems that there is no community consensus over adding generated code to source control or not. I personally try to avoid adding generated code to source control. In this case, that could be solved as part of the build step, no?

Thanks for testing it and the feedback @schmidsi !

I personally don't like submodules.

Same! We had some issues making them work with the cli too. But I think there is no way around it with foundry, since it's the way it manages its deps.

packages/nextjs/generated/deployedContracts.ts is marked as changed.

This was changed after you run yarn deploy right? That file contains "the artifacts" (address + abi) of the contracts deployed with SE-2, so you need them in order to interact with those contracts in the frontend. Doing it in the build step will be tricky since the source of truth lives on the hardhat package (when you deploy the front-end you only deploy the nextjs package). See this discussion also: //github.com/scaffold-eth/scaffold-eth-2/issues/369

I don't want this branch to diverge too much from main, so I think it's a good practice to merge from main once in a while.
The merge it's pretty straightforward (merge from the UI) if non-tricky files are changed (any hook, components, etc) but you have to do it in your machine if there are files like package.json, yarn.lock, template files etc modified. In this case, it was easy, just "accepting ours" for those two files.

just added the PR https://github.com/scaffold-eth/scaffold-eth-2/pull/432 to remove unused types

New npm version published with #432 and #430

Was trying current version of CLI to check how the arguments work, in order to explain them in the Readme, and I'm getting some errors on both my Windows desktop and laptop.
E:\GitHub>npx create-eth@latest test22

 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 | Create Scaffold-ETH 2 app |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+

? What solidity framework do you want to use? foundry
? Install packages? Yes

  √ 📁 Create project directory E:\GitHub\test22
  × 🚀 Creating a new Scaffold-ETH 2 app in test22
    → Only URLs with a scheme in: file and data are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs.
R…
    📦 Installing dependencies with yarn, this could take a while
    📡 Initializing Git repository
ERROR Error occurred Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file and data are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'
    at new NodeError (node:internal/errors:399:5)
    at throwIfUnsupportedURLScheme (node:internal/modules/esm/resolve:1059:11)
    at defaultResolve (node:internal/modules/esm/resolve:1135:3)
    at nextResolve (node:internal/modules/esm/loader:163:28)
    at ESMLoader.resolve (node:internal/modules/esm/loader:838:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:424:18)
    at ESMLoader.import (node:internal/modules/esm/loader:525:22)
    at importModuleDynamically (node:internal/modules/esm/translators:110:35)
    at importModuleDynamicallyCallback (node:internal/process/esm_loader:35:14)
    at file:///C:/Users/Pablo/AppData/Local/npm-cache/_npx/85f97c58a7d429b9/node_modules/create-eth/dist/cli.js:197:63 {
  code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME',
  context: [Object: null prototype] {}
}
Uh oh! 😕 Sorry about that! Exiting...

@Pabl0cks The error might be related to the changes we merged this week, where we started using templates.
Are using NodeJS 18, right?
Could you also try going back to e844021c7754b3c48f236af010a4c85bf8efd78b and test locally? If not, go back to ee958945081ef8429f24dcfe26554be643c60536 and test again. This will help identifying where the bug was introduced.

Are using NodeJS 18, right?

Yep, v18.15.0

Could you also try going back to https://github.com/scaffold-eth/scaffold-eth-2/commit/e844021c7754b3c48f236af010a4c85bf8efd78b and test locally? If not, go back to https://github.com/scaffold-eth/scaffold-eth-2/commit/ee958945081ef8429f24dcfe26554be643c60536 and test again. This will help identifying where the bug was introduced.

Sure! I've went back and the first commit where it started failing is:  2599a95a0428832d9e5154328a317158a3c18666

Sure! I've went back and the first commit where it started failing is: https://github.com/scaffold-eth/scaffold-eth-2/commit/2599a95a0428832d9e5154328a317158a3c18666

That's exactly when we started using templates (using .mjs). After some research:
https://stackoverflow.com/a/70057245

Similar errors in popular libs:
https://github.com/vitejs/vite/issues/1712
https://github.com/okonet/lint-staged/issues/1054
A commit that fixed it on Vite: (not sure if useful for us)
https://github.com/vitejs/vite/commit/33d3cca78607e14e4d0b13e7012d065a94d184b1
@dgrcode Could you take a look?

I've tested it a bit to try to get some extra info - logs from the problem, I think the problem is in that line of code in copy-template-files.ts:
const template = (await import(templateFileDescriptor.path)).default;
Added this code before that line to get some logging:
      const argsFilesPath = extensions
        .map((extension) => {
          const argsFilePath = path.join(
            extensionDict[extension].path,
            argsPath
          );
          const fileExists = fs.existsSync(argsFilePath);
          if (!fileExists) {
            return [];
          }
          writeTextToFile("argsFilePath: "+argsFilePath+"\n",'./output.txt');
          return argsFilePath;
        })
        .flat();

      writeTextToFile("targetDir: "+targetDir+"\n",'./output.txt');
      writeTextToFile("templateFileDescriptor: "+templateFileDescriptor.relativePath.split(templateTargetName)[0]+"\n",'./output.txt');
      writeTextToFile("templateTargetName: "+templateTargetName+"\n",'./output.txt');
      writeTextToFile("templateFileDescriptor.path: "+templateFileDescriptor.path+"\n",'./output.txt');

Those are the last values I get on the log:
argsFilePath: E:\GitHub\scaffold-eth-2-readme-scaffold-config\templates\extensions\hardhat\packages\nextjs\scaffold.config.ts.args.mjs
templateFileDescriptor.path: E:\GitHub\scaffold-eth-2-readme-scaffold-config\templates\base\packages\nextjs.gitignore.template.mjs
templateTargetName: .gitignore
templateFileDescriptor: \packages\nextjs
templateTargetName: scaffold.config.ts
targetDir: E:\GitHub\scaffold-eth-2-readme-scaffold-config\test16
templateFileDescriptor.path: E:\GitHub\scaffold-eth-2-readme-scaffold-config\templates\base\packages\nextjs\scaffold.config.ts.template.mjs
targetDir: E:\GitHub\scaffold-eth-2-readme-scaffold-config\test16
templateFileDescriptor.path: E:\GitHub\scaffold-eth-2-readme-scaffold-config\templates\extensions\hardhat\packages\hardhat.gitignore.template.mjs
templateTargetName: .gitignore
templateFileDescriptor: \packages\hardhat\
Not sure if this helps, let me know if you want to make any other test on Windows.

@Pabl0cks Thanks for this!
Could you try this PR and see if it fixes it? https://github.com/scaffold-eth/scaffold-eth-2/pull/443
P..S I did it a bit blindly, haven't setup a Windows VM (yet :))

P..S I did it a bit blindly, haven't setup a Windows VM (yet :))

This is one of those errors annoying to fix, why couldn't Windows act like the rest of OS 😥
Sent the console error logs I'm getting now in #443

Published 0.0.11 to npm. @Pabl0cks could you test if everything works as expected on windows?
Also added the yarn test alias from https://github.com/scaffold-eth/scaffold-eth-2/pull/445 to both hardhat and foundry.

@Pabl0cks could you test if everything works as expected on windows?

It's working great on windows! gj all 🚀

Closing this in favor of https://github.com/scaffold-eth/create-eth
As we discussed, the CLI version will live in its own repo (GitHub actions already publishing to NPM in that repo)


[macOS] Support setting macOS vibrancy options

Description
Hey, thanks for an awesome project.
Would it be possible to introduce a flag that sets the window title bar color? The color could be specified with a hex code. By default, macOS creates a light grey gradient – it would be awesome to be able to set that to a color that matches the page, enhancing the native look and feel.

@austin-guevara yup, totally possible using Electron options. Passersby, that's an easy thing to add, check the Electron doc and see that PR for an example of adding a similar flag.

Ah, forgot it was already requested in https://github.com/jiahaog/nativefier/issues/35 . @austin-guevara please follow discussion there. Closing.


Charging Station > Enhance firmware upgrade tab to reduce misusage

One of our customer managed to use this feature to upload the firmware and endup in a blocked situation.
Add this new capability in the template and hide/show the corresponding tab in the UI and enforce a backend check:

Add modifications to:

permit to reset the firmware status in DB from the dashboard;
document or enhance the firmware upgrade tab usage:

list firmware files in DB
permit to chose one
permit to choose between HTTP or HTTPS for firmware download with a default to HTTP

In that specific case, remote firmware upgrade is supported by the charging station.
The customer in question have used a feature without knowing how to use it and made a mistake. The fix here is to add an action allowing to reset the firmware upgrade status in the DB to cope with firmware upgrade processus failure or misusage.
In the long run, we also have to deal with charging station feature enablement based on the OCPP key SupportedFeatureProfiles CSL parsing when there's no template available yet or when the feature is not listed in the template. And also control the template features value with that key to trap template errors.

No, again for the nth time, we have separated the capabilities for this reason: we enable them only when tested on our side.
We just follow what we have done so far.
Please proceed.

No, again for the nth time, we have separated the capabilities for this reason: we enable them only when tested on our side.

For the same number of times, it's not a viable solution that already expose a large number of downside:

No automated handling of the number of phases (and the workaround so far require manual changes directly in the DB);
No way to fully test an charging station for the first time:

Template can be done only after the first connection;
Done template need to wait for the next deployment.
To test all the features, we have to make several meetings for no good reason and it's hard to explain to vendors. It make loose time to everybody.

Propositions to deal properly with supported charging stations and unsupported ones have already been discussed at length and accepted to be on the development roadmap since months, minutes here: https://github.com/LucasBrazi06/ev-server/issues/1759
And we are now waiting.

We just follow what we have done so far.

When things done do not match real world usage, they have to be changed.

To come back to that specific issue:

OCPP firmware upgrade is supported by the charging station;
OCPP firmware upgrade is tested and works.

The issue is that in case of failure, the firmware update status cannot be reset is DB from the dashboard. Adding a feature flag for firmware upgrade in template will not solve anything here and is already planned: https://github.com/LucasBrazi06/ev-dashboard/issues/1755
The solution here is:

to permit to reset the firmware status in DB from the dashboard;
to document or enhance the firmware upgrade tab usage:

list firmware files in DB
permit to chose one
permit to choose between HTTP or HTTPS for firmware download with a default to HTTP

No automated handling of the number of phases (and the workaround so far require manual changes directly in the DB);

Nothing to do with the current issue

No way to fully test an charging station for the first time:

Template can be done only after the first connection;
Done template need to wait for the next deployment.
To test all the features, we have to make several meetings for no good reason and it's hard to explain to vendors. It make loose time to everybody.

This is QA testing, the template should be delivered only for QA

Propositions to deal properly with supported charging stations and unsupported ones have already been discussed at length and accepted to be on the development roadmap since months, minutes here: LucasBrazi06/ev-server#1759

Must be summarized in a new issue, cannot go through the whole history again.

When things done do not match real world usage, they have to be changed.

Agree but this is not the case.

@jerome-benoit And stop writing lengthy responses and come to me to discuss each point, it'll be more efficient as you already know!!!

@jerome-benoit And stop writing lengthy responses and come to me to discuss each point, it'll be more efficient as you already know!!!

If from the points developed here and over the previous meetings or exchange and acted to be developed in approved meeting minutes, understanding that:

to test a feature you need to validate, you first need to create the file that will say that the feature can be used and supported (circular dependency, extra unneeded configuration time,  ...)
charging station customer user experience must never be reduced even without official support, just properly stated to him.
(ay least ten more argumented reasons if needed)
is a serious mismatching of real world usage that needs to be solved by following the solution already decided months ago, I do not see how another meeting where they will be repeated once more will change anything. The implementation must just be done now. Period.

Otherwise it's a dupe of https://github.com/LucasBrazi06/ev-dashboard/issues/1755

The enhacements here are really needed, for example on the chargex tenant, they have tried the OCPP fimware update, have failed to use it properly at first and now have a download failed on a charging with no way to reset it themselfves.
I think that issue can stay open with a normal priority.

This feature should have not been enabled at first.
Next if chargeX wants the firmware update this is a dev request + impl + test and after it is enabled in the template.
On the test phase we can enable it in the QA system only.

This feature should have not been enabled at first.
Next if chargeX wants the firmware update this is a dev request + impl + test and after it is enabled in the template.
On the test phase we can enable it in the QA system only.

All the charging stations used on all production tenants have:

OCPP firmware support;
Tested and validated OCPP firmware support by us.

So that specific capabilities support would just have show exactly the same and have lead to exactly the same troubles that that issue try to solve.
And ChargeX people are vendor partner that fully knows how to do firmnware upgrade via OCPP on their own charging stations. They do not need nor want to go through a validation process to use their own station features with a software they pay to supervise them.
With the current dashoard IHM, they have failed to used the firmware upgrade feature.

Firmware update has to be handled by us, firmwares are stored in our DB tested with our backend with no regression...
Again we do not allow CPO to update firmware WE have not tested. This is the goal of the templates.
With this in mind, so far we support nothing expect just Schneider at best as far as I know.
We can discuss this today.


problem with rendering Angular 4 sample app Tour of Heroes

Hi
When I tried to use rendertron and puppeteer to render the Angular 4 sample app Tours of Heroes, the angular 4 ngFor directives are ignored. All the content related to ngFor failed to render.
Due to the async request for heroes, Tour of Heroes actually has 2 stages of loading

load everything except for the ngFor stuff
load the ngFor stuff

It seems that rendertron thinks that after stage 1 of loading, the page has already finished loading. that's why it only renders the stuff without those ngFor async request content.
Since Sam was saying that rendertron can handle all those async requests during Polymer Summit, a fix would be nice~
thx!

Hi @hekod777,
Can you please provide reproduction steps and/or minimized test cases? It is difficult for maintainers of open source projects to be able to help you without these.
Please provide a link to a broken case and what the expected/actual results are.
Thanks,
Sam

Hello @samuelli
here are the steps to reproduce

install angular CLI

npm install -g @angular/cli

clone this repo

https://github.com/johnpapa/angular-tour-of-heroes
3.  cd into the repo
npm install

serve the app

ng serve

then go to

http://localhost:4200/heroes
this is what the site is supposed to show

then start rendertron and enter the following to the input field

http://localhost:4200/heroes

if we click the regular render, the render result will not have those heroes (ngFor)

if we click the shady dom = true render, all the angular 4 components will not be rendered

if we modify src/app/heroes.components.ts line 23 from

getHeroes(): void {
    this.heroService
      .getHeroes()
      .then(heroes => this.heroes = heroes)
      .then(()=>console.log(this.heroes))
      .catch(error => this.error = error);
 }

to
  getHeroes(): void {
    this.heroes = [{id:1, name:'kkk'},{id:2, name:'bbb'}];
  }

rendertron's regular render can render the page just fine.
conclusion, rendertron failed to detect when the angular 4 page actually finished loading when there is an async request.
Please let me know if more details is needed.
thx!
Koddy

these are the steps for angular 4 + polymer 2.0

clone this repo

https://github.com/hekod777/angular4poly2

cd into the folder

npm install

start the angular 4 app

ng serve

there are three parts in this html page

an angular 4 component without using polymer 2.0 web component
an angular 4 component using a polymer 2.0 web component
an polymer 2.0 web components outside of app-root tag

copy http://localhost:4200/ to the input field of rendertron

if we use regular render, it will render all the angular 4 components while ignoring shadow doms

if we use shadydom=true to render the page, all three parts mentioned above will be ignored.

Please let me know if more details is needed.
thx!
Koddy

@samuelli
hmmmm all worked~

Angular 4 async call problem

reason why the tour of heroes ngFor didn't was NOT because of async call. it's because it used InMemoryWebApiModule which is a fake async call for demos and tests. after replacing the fake call with a real async call, everything worked!

Angular 4 + Polymer 2.0

I just do not know why rendertron failed to render my sample app. instead of using the sample app i built, I simply insert Polymer 2.0 elements into tour of heroes app. and everything worked!
Even if I tried my best to mimic my sample app to tour of heroes, and still couldn't get it to work. Maybe just like Donald Duck said "It's fate!"
Anyway, Rendertron is Awesome! If the first time rendering speed can be faster, then it will be perfect!

Great to hear that you got it all working!

@hekod777 can you share the complete repo with Rendertron usage. I am also looking for server side rendering of angular 4 app.

@karunya2530 I am sorry that I am not allowed to share the repo. but it's pretty much the same as the official repo except I added redis for caching and added the setup for downgrading to Node 6 compatible.
other than the repo the AWS configuration was kind of tricky as well.
I am willing share any of my knowledge. Just let me know what kind of questions you have.

Okay. Thanks for the reply. What version on node are you using ? Secondly i have setup an instance of docker on my server with running rendertron on port 8080. Now in my angular app i have installed express middleware and define the index.js. So could you help me out how to define index.js and where to execute index.js on server? Also if i am accessing my site then how rendertron will work for rendering html structure and seo.

hmmmm....

Due to the company policy, I am running rendertron in a docker container using node 6 base image.

My understanding is that you have an express server for your angular app. Then all you need is to use the rendertron middleware in your server/index.js. For example:

// if you want to overwrite the default bot list
const botUserAgents = [
  'W3C_Validator',
  'baiduspider',
  'bingbot',
  'googlebot',
];

const botList = new RegExp(botUserAgents.join('|'), 'i');

app.use(rendertron.makeMiddleware({
  // put in your rendertron endpoint
  proxyUrl: 'https://render-tron.appspot.com/render',
  userAgentPattern: botList,
}));

rendertron middleware will figure out if a request is from a bot based on the user agent. If it's a bot on the bot list, then it will make a request to the rendertron endpoint and send back the SSR version of the page.

ps: We are planning not to use rendertron middleware. Instead we want to use DNS to identify bots and redirect the request to rendertron endpoint. While letting the DNS to verify the true identity of a bot if possible. The pro is that this solution can be used for all kinds of different backends.
pps: for googlebot you can read about this article written by Sam Li. https://medium.com/dev-channel/polymer-2-and-googlebot-2ad50c5727dd

Can anyone share a working example of rendertron with existing angular project?
What changes needs to be done?
If anyone can share a video it would be great. I have searched youtube and got various videos but those all asks for firebase. I want it without firebase.

I just watched the youtube video to deploy rendertron to firebase. This is what I felt.

It didn't use rendertron middleware. Instead it kind of wrote its own rendertron middleware logic. I am not quite sure if it's needed since he was using express server anyway. Maybe that's because the way he handles caching.

Back then we still haven't discovered the share memory size issue for rendertron running in a docker container. Simply the shared default memory size in a docker container is 64mb which is not enough for headless chrome. if it gets many requests the docker container will crash. To deploy rendertron docker container you should consider adding 1. shm-size flag and 2. -security-opt flag. you can find out more details in rendertron's readme.

Let's go back to your questions.

There can be lots of different ways to use rendertron depending on the host, the way you handle cache, and the way you detect bots and make request to rendertron endpoint. Where do you want to deploy rendertron? How do you want to handle cache? How do you want to handle bot detection and making request to rendertron endpoints? If you can have answers for these questions, it should be easier for people to help you on the details on how to configure rendertron for your need.

These are some of the changes and special configurations I have done for rendertron

I am gonna deploy rendertron to AWS and use redis for caching

I wrote my own caching logic and use ioredis to handle redis related transactions.

I implemented babel to transpile rendertron to node 6 compatible and wrote a polyfill for a node 7 function. So that we can use our existing Node 6 base image for rendertron.

I modified the dockerfile to use our Node 6 base image

For AWS configuration, we set the docker container's default share memory size to 2GB because AWS docker configuration doesn't support --shm-size flag at the moment. And implemented --security-opt flag.

We are also working on the DNS configuration for bot detection and request redirection.

The process might be very different if you are going for a different way of deploying rendertron.
It's a good idea to figure out what you are aiming for (the 3 questions above) first.

Hi,
Just wanted to know what changes I should do on my angular project so it
uses rendertron.
On Wed, Mar 21, 2018 at 11:52 AM, hekod777<EMAIL_ADDRESS>wrote:

I just watched the youtube video to deploy rendertron to firebase. This is
what I felt.

It didn't use rendertron middleware. Instead it kind of wrote its own
rendertron middleware logic. I am not quite sure if it's needed since he
was using express server anyway. Maybe that's because the way he handles
caching.
2.
Back then we still haven't discovered the share memory size issue for
rendertron running in a docker container. Simply the shared default memory
size in a docker container is 64mb which is not enough for headless chrome.
if it gets many requests the docker container will crash. To deploy
rendertron docker container you should consider adding 1. shm-size flag and
2. -security-opt flag. you can find out more details in rendertron's readme.
Let's go back to your questions.

There can be lots of different ways to use rendertron depending on the
host, the way you handle cache, and the way you detect bots and make
request to rendertron endpoint. Where do you want to deploy rendertron? How
do you want to handle cache? How do you want to handle bot detection and
making request to rendertron endpoints? If you can have answers for these
questions, it should be easier for people to help you on the details on how
to configure rendertron for your need.
2.
These are some of the changes and special configurations I have done
for rendertron

I am gonna deploy rendertron to AWS and use redis for caching
I wrote my own caching logic and use ioredis to handle redis related
transactions.
I implemented babel to transpile rendertron to node 6 compatible and
wrote a polyfill for a node 7 function. So that we can use our existing
Node 6 base image for rendertron.
I modified the dockerfile to use our Node 6 base image
For AWS configuration, we set the docker container's default share
memory size to 2GB because AWS docker configuration doesn't support
--shm-size flag at the moment. And implemented --security-opt flag.
We are also working on the DNS configuration for bot detection and
request redirection.
The process might be very different if you are going for a different way
of deploying rendertron.
It's a good idea to figure out what you are aiming for (the 3 questions
above) first.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/GoogleChrome/rendertron/issues/94#issuecomment-374843228,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AVM2uSwZPZBzBiw-s4jFMk1x9LxVGmpQks5tgfGdgaJpZM4PKpsF
.

--
Thanks and Regards,
Sunny Bhagat
Follow us on Facebook: https://www.facebook.com/kreativemachinez/

If you are using an express server for your angular app, you should add the rendertron middleware in your server/main.ts file. It will Identify bot requests and making requests to rendertron endpoints.
If you are using some other ways to identify bots and redirecting requests, then you don't need to make any changes to your angular 4 app.

Can we use rentertron middleware for angular 1.4.
I am facing issue in rendering my page.


Add Documentation to Signup Endpoint

Idea
At this point, we have a ton of endpoints and the number just keeps growing. We need to document all of these endpoints but the only way to do that is to write about each of them. That is where you come in. If you would like to document this endpoint, just comment on the post, and I will assign it to you.
Todo

[ ] Add an item to the OpenAPI file that shows how to "Signup"
[ ] Add a documentation page about "Signup"

@William-McGonagle
Hello. Can I take this issue?
I would be happy to help!

Hey @mahiro72,
Sure- we would love the help!

Hey @William-McGonagle

Since the OpenAPI file is in .gitignore
Cannot add...😢


accessing environment variable, easily

"foo"|env
"foo"|env("-")
env("foo")

creating additional module?


Fix keyword arguments for ruby3.0

Is the CI not working?

rake version 10..

sigh
It looks like we need to upgrade the rake version.

No