forge fmt does not make the right line breaks to preserve print width

Component
Forge
Have you ensured that all of these are up to date?

[X] Foundry
[X] Foundryup

What version of Foundry are you on?
forge 0.2.0 (58a2729 2023-05-14T00:16:25.254741000Z)
What command(s) is the bug in?
forge fmt
Operating System
macOS (Apple Silicon)
Describe the bug
there are a few instances of this but basically running forge fmt does not always make the correct choices to format the file according to the single configuration i have set, which you can see here from my foundry.toml:
[fmt]
line_length = 100

to reproduce this bug, you can use this forge version i have specified along and execute forge fmt against this file:
// SPDX-License-Identifier: MIT
//  _____     _ _         _         _
// |_   _|_ _(_) |_____  | |   __ _| |__ ___
//   | |/ _` | | / / _ \ | |__/ _` | '_ (_-<
//   |_|\__,_|_|_\_\___/ |____\__,_|_.__/__/

pragma solidity ^0.8.18;

import {OwnableUpgradeable} from<EMAIL_ADDRESS>import {ReentrancyGuardUpgradeable} from
<EMAIL_ADDRESS>import {AddressResolver} from "./AddressResolver.sol";

/**
 * @dev This abstract contract serves as the base contract for many core
 *      components in this package.
 */
abstract contract EssentialContract is
    ReentrancyGuardUpgradeable,
    OwnableUpgradeable,
    AddressResolver
{
    function _init(address _addressManager) internal virtual override {
        ReentrancyGuardUpgradeable.__ReentrancyGuard_init();
        OwnableUpgradeable.__Ownable_init();
        AddressResolver._init(_addressManager);
    }
}

you (should) notice that line 10 is formatted correctly, but line 9 is not. line 9 should have made a line break after from, like was done on line 10.
this is just one example, but there are a few files that do not format properly / respect the print width configuration i have. let me know if this is not reproducible or if i can share more examples--thanks!

Closing in favor of https://github.com/foundry-rs/foundry/issues/4450 since this is expected behavior
Line 9 in this example has a length of 101, just one character over the line length, and line length is not a hard requirement as discussed in As mentioned in https://github.com/foundry-rs/foundry/issues/4021#issuecomment-1448701684. This is common in formatters, e.g. from the prettier docs:

Prettier’s printWidth ... is not the hard upper allowed line length limit. It is a way to say to Prettier roughly how long you’d like lines to be. Prettier will make both shorter and longer lines, but generally strive to meet the specified printWidth.


Issue with styling fullCalendar is not a function

I'm submitting a ...

bug report

Do you want to request a feature or report a bug?

bug

What is the current behavior?

Application is loading on webpack-dev-server but some elements do not appear in the console. I have error regarding fullCalendar.
vendor.bundle.js:72764 ORIGINAL EXCEPTION: TypeError: jQuery(...).fullCalendar is not a functionBrowserDomAdapter.logError @ vendor.bundle.js:72764
vendor.bundle.js:72764 ORIGINAL STACKTRACE:BrowserDomAdapter.logError @ vendor.bundle.js:72764
logs attached
localhost-1470679273901.txt

** steps to reproduce**

On launching of the page

What is the expected behavior?
loading of the missing elements

What is the motivation / use case for changing the behavior?

Please tell us about your environment:
-Node Version v4.4.7
-npm Version v3.10.6
-Environment Windows 10
-Browser Chrome
-IDE Visual Studio Chrome with Chrome Debugger Extension

Angular version: 2.0.0-rc.X

Other information (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. stackoverflow, gitter, etc)

While in Visual Studio Code I have cannont find name "require" and cannot find name TypeScript JQuery while I have the module installed in node_modules folder.

Hey @IsmaelKP, probably this is related? https://github.com/akveo/ng2-admin/issues/156

Hey, It was probably. I finally deinstall reinstall the project from scratch and it did work. I have no clue on the why unfortunately. I close the issue


Error when running the example

I got this error when running your example
SkeletonUI-macOS-iOS[1728:47422] Task <5ED5FF93-B549-451D-88B1-A54648BCBDA0>.<1> finished with error [-999] Error Domain=NSURLErrorDomain Code=-999 "cancelled" UserInfo={NSErrorFailingURLStringKey=https://rickandmortyapi.com/api/character/, NSLocalizedDescription=cancelled, NSErrorFailingURLKey=https://rickandmortyapi.com/api/character/}
Can you give it a look?
Thanks

It should be fixed in the latest commit both in develop & master branches


Warn when reading discontinuous EDF+ files

Due to the way EDF+ implements the discontinuous extension, a naive EDF reader can still open the file but treats the signals as continuous. This is a potential footgun -- I only noticed it because the annotations referred to times that were much later than the concatenated signals referred to because the annotations assumed that the discontinuity was correctly handled.

we could use this check:
function is_functionally_contiguous_after_start(file::EDF.File)
    # EDF+ says: first annotations signal is such that the first
    # annotation in each list is the start time of the record
    time_idx = findfirst(x -> isa(x, EDF.AnnotationsSignal), file.signals)
    time_anns = file.signals[time_idx]
    starts = [anns[1].onset_in_seconds for anns in time_anns.records]
    # if the start of the next record is exactly seconds_per_record + the start of the previous record
    # then the file is "functionally" contiguous
    return all(==(file.header.seconds_per_record), diff(starts))
end

When this holds, the start may not be at the header's start, but after that all the records are contiguous and reading works. To get the signal start, you need
function start_offset(edf::EDF.File)
    i = findfirst(s -> s isa EDF.AnnotationsSignal, edf.signals)
    i === nothing && return Nanosecond(0)
    annos = edf.signals[i]
    # Shouldn't happen per the spec but who knows, nobody seems to care about the spec
    (isempty(annos.records) || isempty(first(annos.records))) && return Nanosecond(0)
    tal = first(first(annos.records))
    return Nanosecond(tal.onset_in_seconds * 1e9)  # convert to ns since may be non-integer
end

Then when mapping to Onda format, the header's start corresponds to the recording start, and the start_offset corresponds to the signal span's start.


order sets by published date

This replaces the ability to sort sets by their created date with the ability to sort sets by their published date.  This will make management of sets easier for admins.

A datetime column, published_at, is added to the source_sets table.
The value for the published_at column is set to the current value of the created_at column if sets are already published.  This will preserve the current sort order in production.  We do not currently have rspec tests for migrations (maybe something to look into for the future), but I tested this locally to ensure that it works as expected.
A new before_save method is added to the SourceSet model to handle persistence of the publication timestamp.  If the set is being published upon save, the publication timestamp is set to the current datetime.  If a previously-published set is being otherwise edited, the publication timestamp is not changed.  If a set is being un-published, the publication timestamp is cleared.
The order_by method in SourceSet is changed to sort by published_at instead of created_at.

This addresses ticket #8236.

Thanks, @AudreyAltman. My only comment is on the method name published_set. I'm happy to merge this pending your response to that.

@markbreedlove the method name published_set has been changed to check_publish_date as per your suggestion.  Should be ready to :merge:.


Unify Model Interfaces

What behavior of the library made you think about the improvement?
Outlines supports 5 inference engines (Ignoring OpenAI / Azure). We should ensure their interfaces and behaviors are identical. We are accumulating technical debt because of their inconsistencies.
For example, I'm hesitant to implement https://github.com/outlines-dev/outlines/issues/644 / https://github.com/outlines-dev/outlines/issues/647 because I would have to implement it for three distinct interfaces. Another example is the fact that https://github.com/outlines-dev/outlines/issues/614 doesn't work with vLLM or LlamaCpp. Additionally, outlines.generate.regex_llamacpp exists to enforce a sampler constraint on llamacpp, however we have a bug resulting from the fact that outlines.generate.json_llamacpp doesn't exist and doesn't enforce the same sampler constraint.
A breakdown of the discrepancies follows.
Logit modification:

For Transformers / ExLlamaV2 / Mamba logits are modified via logic in SequenceGenerator utilizing a passed FSM.
For LlamaCpp logits are modified via logits processors defined in outlines.models.llamacpp.
For vLLM logits are modified via logits processors defined in outlines.serve.vllm. vLLMs logits processors repeat substantial portions of LlamaCpps logits processors.

Sequence Generation:

For Transformers / ExLlamaV2 / Mamba model() generates the logits, and SequenceGenerator samples and manages the generation.
For LlamaCpp, the model is functionally the SequenceGenerator, and outlines.generate.regex_llamacpp patches the model to apply logits processors. SequenceGenerator is not used.
There is no outlines.models.vllm. Outlines provides a web server and the logits processors. vllm does all the work.

Tokenizer:

Transformers / ExLlamaV2 / Mamba use a TransformersTokenizer
LlamaCpp uses a custom tokenizer compatible with outlines.models.tokenizer.Tokenizer
vLLM patches the tokenizer within the logits processor to be compatible

How would you like it to behave?
Unification should look as follows:
Step 1: https://github.com/outlines-dev/outlines/pull/676

Ensure LlamaCpp and vLLM use the same logits processors
Ensure LlamaCpp and vLLM both use a tokenizer subclassing outlines.models.tokenizer.Tokenizer

don't monkeypatch vLLM tokenizer in the logits processor itself

Result: logits processors have a single canonical implementation
Step 2:

Ensure sequence_generator applies LogitsProcessors, and doesn't manage FSMs

The appropriate logits processor is passed based on the outlines.generate function

Result: all 5 models use logits processors, the business logic for constraining the next token based on an FSM is encapsulated in one place
Step 3:

Implement an abstract model with interfaces defined such that they're compatible with SequenceGenerator

Step 3a:

Implement outlines.models.vllm

outlines.models.vllm.__call__ returns logits and kv cache, and can be used with SequenceGenerator

outlines.models.llamacpp.__call__ returns logits and kv cache, and can be used with SequenceGenerator

Step 3b:

Remove outlines.generate.regex_llamacpp (same for other outlines.generate.*_llamacpp), interface should work with outlines.generate.regex

Involves subclassing llama_cpp.Llama such that llama.Llama.generate() doesn't call Llama.sample() but instead allows Outlines to handle the sampling logic.

Result: All 5 models compatible with SequenceGenerator, model agnostic outlines.generate, outlines.models.vllm available
Step 3c:

Inference engine integration tests: every inference engine X every sampler (related: beam search doesn't work with ExLlamaV2)

Step 4:

"Serve Any Model": https://github.com/outlines-dev/outlines/issues/655

Result: Any model servable via outlines.serve

Additionally, outlines.generate.regex_llamacpp exists to enforce a sampler constraint on llamacpp, however we have a bug resulting from the fact that outlines.generate.json_llamacpp doesn't exist and doesn't enforce the same sampler constraint.

What bug? outlines.generate.json is the default function so it should work for both Transformers and LlamaCpp.

For Transformers / ExLlamaV2 / Mamba model() generates the logits, and SequenceGenerator samples and manages the generation.
For LlamaCpp, the model is functionally the SequenceGenerator, and outlines.generate.regex_llamacpp patches the model to apply logits processors. SequenceGenerator is not used.

The difference between llama.cpp and transformers was temporary; it was quicker to integrate via logits processors than fixing the existing implementation. Eventually it will use SequenceGenerator.

There is no outlines.models.vllm. Outlines provides a web server and the logits processors. vllm does all the work.

That could be easily added. However all model calls in Outlines are synchronous for now, so we wouldn't benefit from vLLM's continuous batching feature.

Ensure LlamaCpp and vLLM use the same logits processors
Ensure LlamaCpp and vLLM both use a tokenizer subclassing outlines.models.tokenizer.Tokenizer

Aren't you moving complexity from one part of the codebase to another? As long as transformers, llama-cpp-python and vLLM use different interfaces for their tokenizer there will be some special casing.

Result: all 5 models use logits processors, the business logic for constraining the next token based on an FSM is encapsulated in one place

I am not opposed to using logits processors in sequence_generator instead of having it manage the FSM. However I doubt this will solve anything? The OpenAI API encapsulates the text generation process, so do vLLM and llama.cpp (for now). You will always need to dispatch depending on the model at the generation level.
We can always "clean" the generate module by dispatching in the way aesara does dispatch Ops for different backends. That is the dispatching logic is implemented where the model is instead of closer to the generation. It comes with trade-offs, but I think it would indeed be cleaner.

outlines.models.vllm.__call__ returns logits and kv cache, and can be used with SequenceGenerator
outlines.models.llamacpp.__call__ returns logits and kv cache, and can be used with SequenceGenerator

Ok if there's an easy way to do this.

Remove outlines.generate.regex_llamacpp (same for other outlines.generate.*_llamacpp), interface should work with outlines.generate.regex

It does as far as the user is concerned.

Involves subclassing llama_cpp.Llama such that llama.Llama.generate() doesn't call Llama.sample() but instead allows Outlines to handle the sampling logic.

That's planned. Again, using sample was a temporary fix for problems that many users were having. There are subtleties linked to KV-cache management when it comes to integrating llama-cpp-python and that will require some work.

Result: All 5 models compatible with SequenceGenerator, model agnostic outlines.generate, outlines.models.vllm available

How does that work with OpenAI?

Inference engine integration tests: every inference engine X every sampler (related: beam search doesn't work with ExLlamaV2)

I'm fine with that, but we'll need to do something to handle the ever-growing test suite duration.


Cheatsheet PDF link is broken

The link to new v.5.0 cheatsheet in PDF is broken:
https://www.raylib.com/cheatsheet/raylib_cheatsheet_v5.0.pdf

@milen-prg Yeah, I'm aware, feel free to send a PR.

Added cheatshet.


Not able to add the integration.

Used HACS to do the download however when trying to complete the last step (adding the integration). It simply exits upon clicking the eskom integration.

I will take a look at this ASAP. I have not been able to test adding the integration via the default repos in HACS since it just got added, but it looks like it got lost again in the PR for one of my other integrations.
I'll get it added back again and see if I can replicate what you are seeing.

@Cerbone the integration is now added back to HACS, could you please uninstall the integration, restart your Home Assistant instance, and go through the install procedure again?
I have just tested installing the integration again and it seems to work as expected.

Having the same issues on my side.

@w1tw0lf can you please check your Home Assistant logs for any errors or warnings from the integration and share them here?
I will do some more testing this evening.

Log Details (ERROR)
Logger: homeassistant.config_entries
Source: config_entries.py:531
First occurred: 2:16:50 PM (1 occurrences)
Last logged: 2:16:50 PM

Error occurred loading configuration flow for integration eskom_loadshedding: No module named 'aiohttp_retry'

Log Details (ERROR)
Logger: homeassistant.config_entries
Source: config_entries.py:531
First occurred: 2:16:50 PM (1 occurrences)
Last logged: 2:16:50 PM

Error occurred loading configuration flow for integration eskom_loadshedding: No module named 'aiohttp_retry'

Thanks for this. I did something illegal with the aiohttp_retry requirement here. I will address it this evening :+1:

Perfect. Great to see some local integrations for HA

@Cerbone @w1tw0lf - I have released v1.0.1 that should address this problem, I've tested it on a clean build and it seems to work now.
You should be able to update in HACS, restart HA, and add the integration. Let me know if it works for you.

Awesome !
Working perfectly on myside.

Great! If @Cerbone can confirm it works I will close this issue.

@swartjean works like a charm. Now I just need to wait for load-shedding to be implemented to see if it updates from 0.


focusing back on previous target region

Hey @astopy whenever you have a free moment can you pause this subjects. The challenge is mostly over and since the classification rate has dropped significantly it would be good to go back to the smaller target region to try and finish that up first. Can you please pause those subjects that have associated name entry with these HiRISE filenames:
PSP_002399_1025
PSP_002544_1025
PSP_002900_1025
PSP_003322_1025
PSP_002758_0945
PSP_002759_0855
PSP_003179_0945
PSP_003180_0945
PSP_003181_0855
PSP_003363_0945
PSP_003364_0945
PSP_003614_0945
PSP_003680_0945
PSP_003970_0945
PSP_004062_0945
PSP_004181_0945
PSP_004748_0945
PSP_004880_0945
PSP_002567_0955
PSP_002936_0955
PSP_003068_0955
PSP_003213_0955
PSP_003424_0955
PSP_003824_0845
PSP_003912_0955
PSP_004202_0955
PSP_004941_0955
PSP_005046_0955
PSP_003345_0955
PSP_003982_0845
No rush, whenever you're free and have time to do this is fine by us. Many thanks for all the help. Happy Holidays and Happy New Year!

Sure, I've paused those again now.


Refactor into an NPM library

Features
Chess Engine
something like:
let chess = new Chess()
chess.setFen(fenString) -> sets the fen string (only the position, not other metadata)
chess.validMoves() -> Move[] returns all valid moves (accounting for the color to move etc.)
chess.fen() -> returns the current fen
chess.execute(move: Move) -> Move.from = SquareID, Move.to = SquareID, Move.castle = boolean, etc. (already done) executes the move passed (updates the fen, color to move, saves the moves in history)
chess.history() -> returns the game history (all the moves played) FullMove[] -> FullMove.white = HalfMove, FullMove.black = HalfMove, HalfMove.position = fen, etc.HalfMove.move = Move
chess.colorToMove() -> Color ("white" | "black")

Chess UI for React
<ChessBoard
  position={fenString} (default starting pos)
  validMoves={Move[]} (default empty) (nothing can move, when provided, clicking on a piece shows valid moves, pieces can be dragged there)
  colorToMove={Color} (default "white")
  size={} -> any valid CSS width/height value (board is always a square) (has a default)
  orientation={"white" | "black"} -> for "white" white on bottom, for "black" black on bottom, default "white"
  blackSquare={} -> any valid CSS bg value (default "#000" or whatever)
  whiteSquare={} -> same as above
  moveIndicator={} -> React.Node -> custom move Indicator (defaults to a little circle in the middle)
  ...otherShit -> decide as necessary (onSquareClick, onSquareDrag,  onPieceMove, etc.)
/>
// Used for showing the player above the board with their takes
<ChessPlayer
  name={string}
  image={image}
  color={Color}
  history={FullMove[]}
/>
// Used for showing the game history beside the board, navigate through moves
<ChessHistory
   history={FullMove[]}
   onPreviousMove={} -> function to call when the previous move is called
   onNextMove={} -> function to call when the next move is called
   onLatestMove={} -> function to call when the latest move is called (goes all the way to the latest half move)
/>

Close #2


Add gauge hydrograph to timeseries plot

If gauge present, pulldata from NWIS web and plot. Also can be used to get Q for sample without measured Q at tiem of smaploing

This is going to be harder than first realized and requires additional dependencies. Delaying for now


Proxy path error

After upgrading from 4.1.1 to 4.1.9, I'm getting the following error:
RuntimeError: Path features/test/index.html proxies to unknown file templates/layouts/features.html.erb
This was working fine in previous versions and wasn't sure if it might be related to https://github.com/middleman/middleman/issues/1846 but it's failing with only two resources in the data file. Thanks for any help!
Here's my snippet:
data.features.each do |feature|
  proxy "/features/#{feature.name.downcase.gsub(" ", "-").gsub(".", "-")}/index.html", "/templates/layouts/feature.html", :layout => "features", :locals => { :feature => feature}
end

🐛

Can you share your full config and directory structure (tree .). A reproducible test case would be helpful as well

Here's the config.rb:

@zslabs Were you able to solve this? If it's similar to the problem I ran into, you'll need to rename features.erb to features.html.erb to get it to work with the latest middleman v4.

Hey @hayley,
Nope, unfortunately still getting path errors. I believe I've tried most every naming combination I can (.html, .html.erb, .erb) in both the actual filename and the proxy reference.

Noticed that if I move the proxy layout files to a path outside of where they were previously /templates/layouts things seem to work okay. @tdreyno was this changed in newer versions or a possible bug?


wesnoth: 1.14.12 -> 1.14.13

Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build wesnoth

Builds and runs on 20.03. I tested the tutorial.
nixpkgs-review pr 91321 shows no errors.


Edited NPS rating options from 4 to 3. (for docs.mattermost.com)

Edited rating options from four options to three.
Updated to add GIF demonstrating NPS widget behavior to capture user feedback on the documentation site.

Thanks @justinegeffen! The change itself looks good (reducing to three emoji), but I assume we'll use this PR to fix the other two issues we found after pushing the previous PR live?

Thanks @justinegeffen! The change itself looks good (reducing to three emoji), but I assume we'll use this PR to fix the other two issues we found after pushing the previous PR live?

Yep, that's the plan. I have raised a Jira ticket for @asaadmahmood as discussed and I believe it'll be actioned in this sprint.

Thanks so much @asaadmahmood, this is super!
Two questions/comments:

Is it possible to add the ability to close the widget without selecting a rating?
Is it possible to add hover text or a tooltip to the button? If, however, we can do option 1, we likely won't need option 2 because the user won't be "stuck" in the rating window.

Hope this makes sense and comments aside, I'm super excited about this and really appreciate all the work you've done. :)

Some notes about the video:

The pop-up opens on click (not hover) but color is highlighted on hover.
Video is (unfortunately) low quality due to GitHub size restrictions so the response looks slow but is very smooth on the site.

Thanks all, this is an exciting change. Looping @amyblais in for visibility.
@asaadmahmood, how much work would be involved in addressing @jasonblais's feedback about the "x"? If not too much, I could create a Jira ticket for it? Would add less overhead to the next phase imo.

I'll fix the x, no problem.

@justinegeffen @jasonblais Fixed the x overlap, we can add the tooltip later.

Looks awesome, thanks @asaadmahmood!

Sweet, thanks @justinegeffen and @asaadmahmood!

@asaadmahmood @justinegeffen Wondering if we'd be good to merge and ship live?
@justinegeffen is there a place where I can check the ratings while you're out early next week?

@jasonblais, I'm happy to ship it so we can start getting event data in. The event data should become available under Reports > Behavior > Events.

Great! Given Asaad reviewed and made changes, I'll go ahead and merge.


HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from th…

Clean backport

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
11m 40s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s
The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ branch-3.1 Compile Tests _

+0 :ok:
mvndep
0m 27s
Maven dependency ordering for branch

-1 :x:
mvninstall
1m 34s
root in branch-3.1 failed.

-1 :x:
compile
0m 57s
root in branch-3.1 failed.

-1 :x:
mvnsite
0m 17s
hadoop-client-runtime in branch-3.1 failed.

-1 :x:
shadedclient
4m 20s
branch has errors when building and testing our client artifacts.

-1 :x:
javadoc
0m 10s
hadoop-client-runtime in branch-3.1 failed.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 20s
Maven dependency ordering for patch

-1 :x:
mvninstall
0m  9s
hadoop-client-runtime in the patch failed.

-1 :x:
compile
0m 25s
root in the patch failed.

-1 :x:
javac
0m 25s
root in the patch failed.

-1 :x:
mvnsite
0m  9s
hadoop-client-runtime in the patch failed.

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
xml
0m  3s
The patch has no ill-formed XML file.

-1 :x:
shadedclient
0m 37s
patch has errors when building and testing our client artifacts.

-1 :x:
javadoc
0m 10s
hadoop-client-runtime in the patch failed.

_ Other Tests _

+1 :green_heart:
unit
0m 10s
hadoop-project in the patch passed.

-1 :x:
unit
0m  9s
hadoop-client-runtime in the patch failed.

+1 :green_heart:
asflicense
0m 22s
The patch does not generate ASF License warnings.

21m 33s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1976/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/1976

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml

uname
Linux 5aa7aa65ea71 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
branch-3.1 / 03ff1d3

Default Java
Oracle Corporation-9-internal+0-2016-04-14-195246.buildd.src

mvninstall
https://builds.apache.org/job/hadoop-multibranch/job/PR-1976/1/artifact/out/branch-mvninstall-root.txt

compile
https://builds.apache.org/job/hadoop-multibranch/job/PR-1976/1/artifact/out/branch-compile-root.txt

mvnsite
https://builds.apache.org/job/hadoop-multibranch/job/PR-1976/1/artifact/out/branch-mvnsite-hadoop-client-modules_hadoop-client-runtime.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-1976/1/artifact/out/branch-javadoc-hadoop-client-modules_hadoop-client-runtime.txt

mvninstall
https://builds.apache.org/job/hadoop-multibranch/job/PR-1976/1/artifact/out/patch-mvninstall-hadoop-client-modules_hadoop-client-runtime.txt

compile
https://builds.apache.org/job/hadoop-multibranch/job/PR-1976/1/artifact/out/patch-compile-root.txt

javac
https://builds.apache.org/job/hadoop-multibranch/job/PR-1976/1/artifact/out/patch-compile-root.txt

mvnsite
https://builds.apache.org/job/hadoop-multibranch/job/PR-1976/1/artifact/out/patch-mvnsite-hadoop-client-modules_hadoop-client-runtime.txt

javadoc
https://builds.apache.org/job/hadoop-multibranch/job/PR-1976/1/artifact/out/patch-javadoc-hadoop-client-modules_hadoop-client-runtime.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-1976/1/artifact/out/patch-unit-hadoop-client-modules_hadoop-client-runtime.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-1976/1/testReport/

Max. process+thread count
127 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-client-modules/hadoop-client-runtime U: .

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-1976/1/console

versions
git=2.7.4 maven=3.3.9

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@jojochuang I got PluginExecutionExceptio in hadoop-maven-plugin when I applied this patch to branch-3.1. cherry-picking 832852ce4ff0 fixed the error.


SynchronizerShiftReg no longer allows sync=0

With pending changes to the synchronizer primitives in https://github.com/chipsalliance/rocket-chip/pull/2212, sync=0 is no longer allowed as a shortcut to bypass the synchronizer altogether.  This PR adds scala code to explicitly bypass the synchronizer in this case.

May I instead propose that we alter https://github.com/chipsalliance/rocket-chip/pull/2212 to continue to handle the special case, as a means to reduce repetition of boiler-plate code
Specifically, the if-statement can be added in the companion object's apply method: https://github.com/chipsalliance/rocket-chip/pull/2212/files#diff-ec029162ec5265bb92da0af9bc7c13cdR115

@mwachs5 I agree with Andrew.  If you also agree, could you update your synchronizer-updates branch to handle the 0 sync stages case?  Then I will close this PR.

Ok, I will make that change only to the API/ helper method part, not the actual module (I do not want an actualy instance of SynchronizerShiftReg that is only depth 0).

Cool, thanks.

@ernie-sifive please give https://github.com/chipsalliance/rocket-chip/pull/2212/commits/bb3d53d1c01528b9decd4028392d427dad83cbd1 a shot

https://github.com/chipsalliance/rocket-chip/commit/bb3d53d1c01528b9decd4028392d427dad83cbd1 still fails because UART is using SyncResetSynchronizerShiftReg.  It seems garbanzo_asic tries to use UARTPinsFromPort with sync=0 whereas I imagine other chip configs may call with a different sync value.
main/scala/everywhere/platforms/PlatformPinMux.scala:      UARTPinsFromPort(uart, sys_uart, clock, reset, syncStages = 0)
main/scala/platform/CLN28HPCPlatform.scala:      UARTPinsFromPort(internal, port, pClk, pReset, syncStages = 3)


Addendum HADOOP-17770 WASB : Support disabling buffered reads in positional reads - Added the invalid SpotBugs warning to findbugs-exclude.xml

NOTICE
Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HADOOP-XXXXX. Fix a typo in YYY.)
For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 54s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

_ trunk Compile Tests _

+1 :green_heart:
shadedclient
21m 50s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1 :green_heart:
xml
0m  1s

The patch has no ill-formed XML file.

+1 :green_heart:
shadedclient
15m 34s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
asflicense
0m 33s

The patch does not generate ASF License warnings.

40m 42s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3223/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3223

Optional Tests
dupname asflicense codespell xml

uname
Linux 634bdf60bb90 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 3b380dfe5c909f167202b64ea6abea8fbae2b736

Max. process+thread count
566 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3223/1/console

versions
git=2.25.1 maven=3.6.3

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 32s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

_ trunk Compile Tests _

+1 :green_heart:
shadedclient
18m 47s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
xml
0m  1s

The patch has no ill-formed XML file.

+1 :green_heart:
shadedclient
13m  1s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
asflicense
0m 35s

The patch does not generate ASF License warnings.

34m 41s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3223/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3223

Optional Tests
dupname asflicense codespell xml

uname
Linux 764c7be2129b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / d6304e06cbd41ec3c24a5cdea5dac82c3bb0128e

Max. process+thread count
696 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3223/2/console

versions
git=2.25.1 maven=3.6.3

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

+1


[AppService] containerApps_ListSecrets operation probably miss a resourceGroup in the path

From the swagger reference here, it doesn't need a resourceGroupName parameter at all, but according to this issue https://github.com/Azure/azure-sdk-for-js/issues/20042 reported from our JS SDK customer, it actually need a resourceGroupName parameter
From the containerApp resource related operations here, it indeed has an resourceGroup here.
We proably need to change the path of listSecrets operation from
"/subscriptions/{subscriptionId}/providers/Microsoft.Web/containerApps/{name}/listSecrets"

to
"/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Web/containerApps/{name}/listSecrets"

But as the issue https://github.com/Azure/azure-sdk-for-js/issues/20042 mentioned, even if we tried to change it this way in our local and regenerate the JS SDK, we still can't get the expected response.  Guess there're also some fixes needed in their service backend.

@qiaozha Redirecting to services team to look into it.


Deprecated Rofi theme definition

First of all thanks for the amazing guide you've put together.
I found an issue with a deprecated theme definition in the Rofi configuration file (config.rasi):
theme: "onedark"; should be replaced with @theme: "onedark" at the end of the file, outside of the brackets.

Fixed here: https://github.com/antoniosarosi/dotfiles/tree/arco/.config/rofi/config.rasi
(Not merged into master yet)


ARROW-9: Rename unchanged "Drill" to "Arrow"

This PR renames the unchanged "Drill" in codes, comments, tests to "Arrow".

@jacques-n @wesm would you take a look at this please? Thanks.

LGTM (assuming test suite passes). Unrelatedly we should start a terminology thread on the ML so that we have standard names for each Arrow concept (for example I used "array" and not "vector" in the C++ prototype)


update package.json to fix handlebars issue

Currently the package.json requires a specific version of handlebars:
"handlebars": "4.4.5",

There is a security issues with handlebars: https://npmjs.com/advisories/1316
Remediation:  Change the line with handlebars in package.json to:
"handlebars": "^4.5.2",

An updated version was published with 4.5.3


Add Health Heartbeat to App Insights SDK

Proposal: Health Heartbeat
We would like to add a health 'heartbeat' to the Application Insights SDK. We believe that by adding this feature to the App Insights SDK we can enable more complete support scenarios for our customers.
The new information that the SDK will provide to the data stream will be very low cost, and have high value for supporting the use of Application Insights within a deployed service or application.  All of the data associated with the health heartbeat would be stored in the customer's data store, adding to their overall monitoring costs but providing extra value with regards to maintenance and support.
Please comment on the proposal below, we are happy to hear feedback from the community!
Proposed new documentation regarding the Health Heartbeat:
Current documentation here

Application Insights Diagnostics Telemetry
The DiagnosticsTelemetryModule works at the Application Insights tooling level exclusively. The two things that it will emit will be (1) reporting errors in the Application Insights instrumentation code itself, and (2) report a 'health heartbeat' to aid in diagnostic efforts.
When reporting errors, for example, if the code cannot access performance counters or if an ITelemetryInitializer throws an exception. Trace telemetry tracked by this module appears in the Diagnostic Search. Sends diagnostic data to dc.services.vsallin.net.+
The health heartbeat monitoring information is set to send a minimal amount of data to track general health information on your application as it would pertain to the Application Insights tooling itself. It will log information like platform versions, SDK versions, and other such data, and is open to extension should your application have specific health-heartbeat requirements.
The data is sent at very infrequent intervals and is by default sent once every n (TBD: determine default n) seconds. All data would be stored with your application telemetry and can be queried and used for support or other purposes later.

Microsoft.ApplicationInsights.Extensibility.Implementation.Tracing.DiagnosticsTelemetryModule
Microsoft.ApplicationInsights NuGet package. If you only install this package, the ApplicationInsights.config file is not automatically created

Proposed Enhancements
We will extend the functionality within the DiagnosticsTelemetryModule to include this health heartbeat functionality. The module have a new set of classes added that will:

Perform heartbeat actions on a timed schedule

The user can configure the rate at which these heartbeat payloads are     sent in the ApplicationInsights.config file.

Read a specific set of information on the running application that will be added as properties to the heartbeat message

This will include adding any extended information that the user has provided that they want appended to the message

We will provide an interface & registration to the user of the system that will allow the addition of any further telemetry to the heartbeat message

This will include pruning out message elements that the user wants excluded
Configuration file elements will be added to allow the user to customize the payload of the health heartbeat to their liking.

Set the status of the message to 0 if all elements are successfully read, and no obvious failure conditions are met

In the provided interface & registration for extending the health heartbeat for users, failure status can also be affected

Send the message to the App Insights component

One metric will be sent to every iKey named in ApplicationInsights.config

Allow the user to completely remove the functionality as well, by removing it from the ApplicationInsights.config file.

The Message & Payload
The message that will deliver this heartbeat will be an Application Insights metric type, where we will set the metric value to 0 for successful health heartbeat delivery, and increment this value by one for each field in the payload that represents a potentially unhealthy status.
The data that will be included in the health heartbeat by default will be the following...
Fields:

Name
Type
Example Value
Description

osType
string
"Windows"
operating system of service/application instrumented

location
string
"EastUS"
string representing the location of the Azure region the VM is running in

name
string
"VM01"
name of the VM running this instance of your application

offer
string
"WIN10"
name of the VM image if it was deployed from the VM image gallery

platformFaultDomain
string
"0"
fault domain the VM image is running in

platformUpdateDomain
string
"0"
update domain the VM image is running

publisher
string
"Microsoft"
publisher of the VM image

sku
string
"10.0"
sku of the VM image

version
string
"7.3.2"
version of the VM image

vmId
string
"59a7f16f-a821-414b-abfe-da0fba8ab6b4"
id of the VM  instance application is running on

vmSize
string
"Standard_DS2"
VM size application is running on

targetFramework
string
".NET Framework 4.6.2"
The target framework the application was compiled against (to the best of our ability)

runtimeFramework
string
".NET Framework 4.6.2"
Runtime framework version down to patch

appinsightsSdkVer
string
"The current Application Insights SDK elements present, and their version numbers

Potential Fields (TBD to include these or not):

Name
Type
Example Value
Description

version
string
"1.0"
The version of the health heartbeat packet 'schema'

time
time
2017-03-22T12:34:59.12345Z
Timestamp of the heartbeat event in UTC

seqnum
int64
1004
index for a specific message enumerating the order of health heartbeat events sent

unhealthyItem
string[]
"userField01","userfield02"
name of any user fields that are reporting 'unhealthy' status

Failure Status Evaluations
Failure status will be set based on the following criteria by default:

A user extension is present that will affect the failure status

Interfaces Available for Extension
IHealthHeartbeatProperty will be implemented by consumers of the Telemetry client and named in the ApplicationInsights.config file.

Users can supply a set of name-value pairs that will be appended to the payload of the health heartbeat message
Users can indicate that any of their properties represent unhealthy conditions have been met in the payload they are adding to the message

Note, the DiagnosticsTelemetryModule will ensure no conflicting property names are appended into the payload. Upon encountering such collisions, the name of the module adding the property will be prepended to the property name. If collisions are still present after performing this, an incremented number will be added as the suffix of the property name until there are no collisions.
The class that implements the IHealthHeartbeatProperty will undergo a registration in the DiagnosticTelemetryModule and will be added to the payload upon successful registration. The user can also register their class during runtime if they wish, but will miss out on any health heartbeat metrics sent prior to calling RegisterHeartbeatPayload.
Configuration File Elements Added
We will add a section to the ApplicationInsights.config file that will allow the users to do the following:

Change the cadence at which the health heartbeat message is sent
Remove specific payload fields that are sent in the health heartbeat message
Name the custom extensions that the user has created to alter/add payload information with, and that must be loaded at runtime

A heartbeat monitor is something I need to implement for what I'm developing at the moment. What's the guesstimate ETA on this?

@aguvenli I'm currently new to the team and learning how to affect the SDK. I would put the ETA on getting a beta version of this in the 2-3 week range, shipped version after that.

Am rethinking the registration of user-defined payload extensions. Instead of naming them in the config file and having them get loaded at runtime via reflection, just have the user register them in their own code instead upon initialization.

Id add my vote to this.  Currently building numerous windows services which id like to monitor their running state, which this would allow.
@d3r3kk the ability to change the user-defined payload during runtime would provide me metrics on failure, i.e. what was the apps state before it doesn't call home again.

@d3r3kk a few comments/questions:
What's the advantage of doing this in the SDK versus letting apps do it themselves with, for example, a periodic timer calling _telemetryClient.TrackEvent("Heartbeat") ?
Is there going to be a special Azure Portal view for heart-beat events? What about support for live stream?
We already have "Availability" events. How does this compare?
How are you getting the values for the Azure-specific payload properties (location/offer/VMsize/fault-domain etc.)? Can they all be queried by running code in the user's app? Or is there some Azure fabric "magic" that can supply them?
I'm wondering whether this belongs in the base SDK at all. Is there anything in the proposal that requires access to base SDK internals? Perhaps it could be delivered as a separate Nuget package.

@pharring can and alert be triggered when an Event doesn't show up after a given time?  I suspect the heartbeat would require specific functionality in the portal to watch the "hits" and respond when they aren't received?

@pharring and @matt-shepherd thanks for the comments. I'll try to answer your queries the best I can, but ultimately @SergeyKanzhelev or @Dmitry-Matveev will have the best ideas for this feature, so I'll let them chime in if they wish as well.

What's the advantage of doing this in the SDK versus letting apps do it themselves...

I believe the idea is to provide everyone with a heartbeat item that they can make use of however they like, without having to implement anything on their own. Of course, we need to keep it simple and extensible while allowing the user full control at the same time.

Is there going to be a special Azure Portal view for heart-beat events? What about support for live stream?

I believe the idea is to support a feature on the portal eventually, but nothing specific has been discussed yet. @SergeyKanzhelev will have a better answer here I am sure. As for Live Metrics Stream, since the proposed payload is simply a MetricTelemetry item we can filter for this heartbeat specifically via the new filtering functionality (available in the LMS today).

We already have "Availability" events. How does this compare?

Availability events, if I am understanding them correctly, currently are only used to measure availability via the WebTest functionality we expose in the portal. I see this as a more of a 'pull' availability function where the proposal here would be more of a 'push' one, and would be generic in terms of type of application. Also, when we were discussing the implementation of this feature we felt that overloading this type might confuse existing UX elements that are looking for this type today.

How are you getting the values for the Azure-specific payload properties...

The intent here is to get the Azure-specific ones (when they are available) from the Azure Instance Metadata Service.

Can they all be queried by running code in the user's app...

Yes, as per the link above.

I'm wondering whether this belongs in the base SDK at all...

I think the primary focus here is the 'you get this for free' and it comes alongside/within the DiagnosticsTelemetryModule that is already in the base. The idea was that heartbeat data is immediately emitted and by default turned on (while still being controllable by the user) to help in user support scenarios, following the same idea behind the DiagnosticsTelemetryModule.

@d3r3kk thanks. Especially thanks for the links to the Azure Instance Metadata Service. I was unaware of it. Of course, that works only on Azure VMs and not on Azure App Service (I tried it) or on-prem machines.
Of course it's up to you what you put in the base SDK. My mental test for what goes there is "Does any reasonable implementation this require internal knowledge of types in the base SDK?" and, as I understand it, this proposal fails that test. Perhaps the fact that it uses the "vsallin.net" endpoint is sufficient grounds, though. I assume that endpoint is special - although I don't know what else it's used for.
I mentioned availability events because it seems like a heartbeat is a sort of availability signal - albeit originating from within the app itself, rather than triggered by an external health-check. I don't know how heavily they're used today, but perhaps they could serve an enhanced purpose.
Anyway, I don't mean to derail the proposal. These are just things to think about. Good luck with the proposal.

Here are some thoughts:

I do not think we'll send anything to vsallin.net and the documentation is probably incorrect on this point. This used to be staging endpoint long time ago but diagnostic module should've been sending to dc.services.visualstudio.com or applicationinsights.io addresses.
We need to be careful with the outgoing calls to fetch Azure information via external services - these calls will be monitored by our own Dependency collection and we might need to exclude those (and also differentiate between customer calls to that service and ours to keep only customer's calls).
If we have a lot of Azure specific information retrieved in this way, we may want to check if other cloud providers expose similar functionality and leverage them / address point (2) for them as well.
If we add OS / Name and other platform info into this event, we have a slight duplication as such information might be already present in all other telemetry items we collect.
It might be better to go with the separate module if we plan to brand is as an extensible heartbeat the customer may want to use and extend instead of the internal heartbeat that our modules will need to use (we discussed it with @d3r3kk offline)

@pharring, the initial idea was to have a metric with custom dimensions for the internal SDK health tracking and heartbeat with the most common properties and add ability for any SDK module (like Quick Pulse) to add to this information and change state of this particular SDK instance from healthy to broken. The higher the value of the metric, the more issues SDK is having. This goes in hand with some internal monitoring tools being widely used in MSFT as far as I know. This slightly differentiates heartbeat approach from the availability telemetry.
With heartbeat, one can answer the questions like the following in our UI or analytics:

Does SDK stop populating data if everything is calm?
Are any of my SDKs misconfigured (does not solve the wrong iKey case :) )?

That should also help support engineers to access the customer environment and SDK states quickly.
There was one more aspect to it (people asked for it offline) which I'm not certain is easy to address - use this telemetry to analyze if some issue has been fixed with the rollout of new SDK or similar self-monitoring approaches. So, that's still under the discussion at this point. Hope this helps!

Thanks @Dmitry-Matveev and @pharring for your feedback and comments. I am still ramping up my knowledge of this SDK and any feedback I can get from folks who know it better serves to make me more efficient at learning the important bits, faster.
I am still putting the code into a branch on my own forked repo to gain a better understanding of the SDK itself as well as the extension and how to best implement. I'm leaning a bit towards an extension module at the moment as including it within the base SDK doesn't seem to lend itself well to configuration that I want exposed.
From @Dmitry-Matveev comments:

I do not think we'll send anything to vsallin.net...

I will put that as an item to resolve when I go about updating the documentation for this feature.

We need to be careful with the outgoing calls to fetch Azure information via external services...

Ah yes I understand. Once I get to the part where I am querying for heartbeat payload data I will pay very close attention to this issue, hopefully I can provide a solution to anyone extending the feature as well in this regard (ie. What if someone wants to ping some website or make a request to some other service as part of the payload? We might want to allow them a simple way to 'mask' those call too?).

If we have a lot of Azure specific information retrieved in this way, we may want to check if other(s)...

Agreed. Will keep that in mind.

If we add OS / Name and other platform info into this event, we have a slight duplication...

Interesting. I will look to see how we gather that information elsewhere then and try and keep things consistent, and perhaps start a conversation on the value of adding that info to this payload.

It might be better to go with the separate module...

As stated above, I'm still evaluating this and leaning somewhat towards the modularization idea. Will have a better idea in the next couple of days.

@d3r3kk You mentioned that this will be done through the DiagnosticsTelemetryModule. Is that a module that the user has to add explicitly to their config, or does it automatically and always get added?
Is the usage pattern such that customers would generally not have this module, but if something goes wrong, they would be encouraged to update their config, add this module and get more health information than they had before?
Does this proposal intend to change how telemetry is collected today? Today, AI SDK usage is monitored through fields (like sdkVersion) in every telemetry item's TelemetryContext. Can we use data gathered by this proposal to understand usage patterns instead?
My motivation for asking these questions is that I am evaluating what is the right option to monitor usage of Telemetry Initializers provided by Application Insights for Service Fabric Sdk. I initially thought about, using the internal Sdk Version field but that field already has a value filled in by the TelemetryModule that generated the telemetry - I don't want to blindly replace it in my TelemetryInitializer.

Health Heartbeat is a very good idea. We need this so much.
Our application ( a .net exe) runs at a lot of organizations non-stop as a windows service. It uses App Insights. Our requirement is finding out if the application stops running at any location. So if on one of our customers' servers application is not running e.g. because of a faulty update, we want to find out as soon as possible so we can respond.
Will this be possible with this feature?

@nizarq DiagnosticsTelemetryModule is added by default.
https://github.com/Microsoft/ApplicationInsights-dotnet-server/issues/220

@d3r3kk
Once we are are done with the initial implementation, I guess you are planning to add other properties like os, vmsize etc. Could you also collect process-id as well as part of this? This was originally discussed here:
https://github.com/Microsoft/ApplicationInsights-dotnet-server/issues/620

Thanks for the great questions all! Let me see if I can answer them...
From @nizarq:

Is the usage pattern such that customers would generally not have this module, but if something goes wrong, they would be encouraged to update their config, add this module and get more health information than they had before?

Actually, my intention is that the health heartbeat would be automatically added (it sends very little data) and if you wish, you can turn it off. The user can control the data that will be sent with each heartbeat in that they can turn fields off if they don't want to see them, or if they would like to reduce data to the smallest possible footprint and still get a heartbeat. Also, the user can opt to add custom fields to the heartbeat pulse as they wish.

Does this proposal intend to change how telemetry is collected today?

No. This is simply an enhancement to what we have today.

Can we use data gathered by this proposal to understand usage patterns instead?

This is not the intention for this feature, no. You can use Application Insights to understand usage patterns in other ways  and with some other features we've provided in Azure.
From @etm-admin:

...if on one of our customers' servers application is not running e.g. because of a faulty update, we want to find out as soon as possible so we can respond. Will this be possible with this feature?

This is one way you can make use of the Health Heartbeat feature, yes. You could set up alerts that detect when an app stops sending the heartbeat, and respond as necessary.
From @cijothomas:

I guess you are planning to add other properties like os, vmsize etc.

Yes, I am learning as I go and getting the feature ready to ship is my initial concern. Adding 'default' fields that the SDK would emit will come second.

Could you also collect process-id as well as part of this?

I think we might be able to accommodate this, it does seem useful and most likely this is the best place for something like this. When I get to implementing the various payload quantities I will investigate & incorporate this. (I've updated the spec to include it so we don't forget).

Heartbeat feature is in 2.5.0-beta2. Closing this issue. Follow-up issues can be created as necessary.


[Feature] refresh materialized view when base tables are refreshed

What type of PR is this：

[ ] bug
[x] feature
[ ] enhancement
[ ] refactor
[ ] others

Which issues of this PR fixes ：

Fixes #6852
Problem Summary(Required) ：

[FE PR Coverage check]
:disappointed: fail : 0 / 10 (00.00%)
file detail

path
covered line
new line
coverage

:large_blue_circle:
com/starrocks/transaction/PublishVersionDaemon.java
0
10
00.00%

[FE PR Coverage check]
:disappointed: fail : 0 / 18 (00.00%)
file detail

path
covered line
new line
coverage

:large_blue_circle:
com/starrocks/transaction/PublishVersionDaemon.java
0
18
00.00%

[FE PR Coverage check]
:disappointed: fail : 0 / 17 (00.00%)
file detail

path
covered line
new line
coverage

:large_blue_circle:
com/starrocks/catalog/MaterializedView.java
0
3
00.00%

:large_blue_circle:
com/starrocks/transaction/PublishVersionDaemon.java
0
14
00.00%

CREATE TABLE tbl2
(
k1 date,
k2 int,
v1 int
)
DISTRIBUTED BY HASH(k2) BUCKETS 3
PROPERTIES('replication_num' = '1');

INSERT INTO tbl2 VALUES('2022-01-26', 2, 2);
INSERT INTO tbl2 VALUES('2022-01-27', 3, 3);
INSERT INTO tbl2 VALUES('2022-02-01', 4, 4);
INSERT INTO tbl2 VALUES('2022-02-02', 5, 5);
INSERT INTO tbl2 VALUES('2022-02-03', 6, 6);

create mv
CREATE MATERIALIZED VIEW tbl2_mv1
DISTRIBUTED BY HASH(`k2`) BUCKETS 10
REFRESH ASYNC
AS
select
    k2,
    sum(k2) as k2_pv
from tbl2
group by k2;

CREATE MATERIALIZED VIEW tbl2_mv2
DISTRIBUTED BY HASH(`k2`) BUCKETS 10
REFRESH MANUAL
AS
select
    k2,
    sum(k2) as k2_pv
from tbl2
group by k2;

CREATE MATERIALIZED VIEW tbl2_mv3
DISTRIBUTED BY HASH(`k2`) BUCKETS 10
REFRESH ASYNC START('2122-12-31') EVERY(INTERVAL 1 HOUR)
AS
select
    k2,
    sum(k2) as k2_pv
from tbl2
group by k2;

insert value
 INSERT INTO tbl2 VALUES('2022-02-13', 7, 6);

check task status
MySQL [information_schema]> select * from tasks;
+-----------+---------------------+----------------------------------------------------+----------+----------------------------------------------------------------------------------------------------------------------+-------------+
| TASK_NAME | CREATE_TIME         | SCHEDULE                                           | DATABASE | DEFINITION                                                                                | EXPIRE_TIME |
+-----------+---------------------+----------------------------------------------------+----------+----------------------------------------------------------------------------------------------------------------------+-------------+
| mv-11137  | 2022-07-12 23:05:56 | PERIODICAL (START 2122-12-31T00:00 EVERY(1 HOURS)) | test     | SELECT `test`.`tbl2`.`k2` AS `k2`, sum(`test`.`tbl2`.`k2`) AS `k2_pv` FROM `test`.`tbl2` GROUP BY `test`.`tbl2`.`k2` | NULL        |
| mv-11043  | 2022-07-12 23:04:15 | MANUAL                                             | test     | SELECT `test`.`tbl2`.`k2` AS `k2`, sum(`test`.`tbl2`.`k2`) AS `k2_pv` FROM `test`.`tbl2` GROUP BY `test`.`tbl2`.`k2` | NULL        |
+-----------+---------------------+----------------------------------------------------+----------+----------------------------------------------------------------------------------------------------------------------+-------------+
2 rows in set (0.01 sec)

MySQL [information_schema]> select * from task_run;
ERROR 1064 (HY000): Unknown table 'task_run'
MySQL [information_schema]> select * from task_runs;
+--------------------------------------+-----------+---------------------+---------------------+---------+----------+----------------------------------------------------------------------------------------------------------------------+---------------------+------------+---------------+
| QUERY_ID                             | TASK_NAME | CREATE_TIME         | FINISH_TIME         | STATE   | DATABASE | DEFINITION                                                                                | EXPIRE_TIME         | ERROR_CODE | ERROR_MESSAGE |
+--------------------------------------+-----------+---------------------+---------------------+---------+----------+----------------------------------------------------------------------------------------------------------------------+---------------------+------------+---------------+
| 2f802a9a-01f4-11ed-8e58-0242e42668fb | mv-11043  | 2022-07-12 23:06:19 | 2022-07-12 23:06:20 | SUCCESS | test     | SELECT `test`.`tbl2`.`k2` AS `k2`, sum(`test`.`tbl2`.`k2`) AS `k2_pv` FROM `test`.`tbl2` GROUP BY `test`.`tbl2`.`k2` | 2022-07-15 23:06:19 |          0 | NULL          |
| 21cca0e7-01f4-11ed-8e58-0242e42668fb | mv-11137  | 2022-07-12 23:05:56 | 2022-07-12 23:05:57 | SUCCESS | test     | SELECT `test`.`tbl2`.`k2` AS `k2`, sum(`test`.`tbl2`.`k2`) AS `k2_pv` FROM `test`.`tbl2` GROUP BY `test`.`tbl2`.`k2` | 2022-07-15 23:05:56 |          0 | NULL          |
| f07e40b9-01f3-11ed-8e58-0242e42668fb | mv-11043  | 2022-07-12 23:04:34 | 2022-07-12 23:04:34 | SUCCESS | test     | SELECT `test`.`tbl2`.`k2` AS `k2`, sum(`test`.`tbl2`.`k2`) AS `k2_pv` FROM `test`.`tbl2` GROUP BY `test`.`tbl2`.`k2` | 2022-07-15 23:04:34 |          0 | NULL          |
| e5399425-01f3-11ed-8e58-0242e42668fb | mv-11043  | 2022-07-12 23:04:15 | 2022-07-12 23:04:15 | SUCCESS | test     | SELECT `test`.`tbl2`.`k2` AS `k2`, sum(`test`.`tbl2`.`k2`) AS `k2_pv` FROM `test`.`tbl2` GROUP BY `test`.`tbl2`.`k2` | 2022-07-15 23:04:15 |          0 | NULL          |
+--------------------------------------+-----------+---------------------+---------------------+---------+----------+----------------------------------------------------------------------------------------------------------------------+---------------------+------------+---------------+
4 rows in set (0.00 sec)

run starrocks_fe_unittest

[FE PR Coverage Check]
:disappointed: fail : 0 / 19 (00.00%)
file detail

path
covered line
new line
coverage
not covered line detail

:large_blue_circle:
com/starrocks/catalog/MaterializedView.java
0
3
00.00%
[501, 502, 503]

:large_blue_circle:
com/starrocks/transaction/PublishVersionDaemon.java
0
16
00.00%
[178, 358, 359, 360, 362, 363, 364, 365, 366, 367, 368, 369, 371, 372, 374, 376]

run starrocks_admit_test


Simplify hints declaration and make frontend responsible for number of outputs, not hint / backend

Creating this one as draft before cleaning it as it opens a discussion. It reverts some changes introduced by #183  #175  (cc @ivokub ).
Essentially, this PR proposes to remove nbInputs and nbOutputs set in StaticHintFunction and the NbOutputs() function in the Hint interface.
Trying to implement multi-outputs (with variable length) hints for binary operation, this declarative way is really not helping;

The circuit developer will provide a list of inputs to the hint function (example; api.NewHint(myHint, input0, input1, ...). IF (and I don't think we need to) we want to check that the number of inputs matches what the hints can process, it's up to the hint to perform the check (ie if len(inputs) > ... --> error). Currently this was encapsulated in a Call function.

Similarly, the circuit developer knows how many outputs it expects from a hint. He should set it like so bits := api.NewHint(BitsHint, 220, value) would return the first 220 bits for example in []bits. Previously this was done declaratively at hint definition, through NbOutputs() method, which was called both in the frontend and the backend at solving time. --> it makes no sense as the backend will not decide the number of outputs, the frontend did at compile time and created the exact number of wires needed.

@ivokub did you have specific future scenario in mind when doing this pattern?

Creating this one as draft before cleaning it as it opens a discussion. It reverts some changes introduced by #183 #175 (cc @ivokub ).
Essentially, this PR proposes to remove nbInputs and nbOutputs set in StaticHintFunction and the NbOutputs() function in the Hint interface.
Trying to implement multi-outputs (with variable length) hints for binary operation, this declarative way is really not helping;

The circuit developer will provide a list of inputs to the hint function (example; api.NewHint(myHint, input0, input1, ...). IF (and I don't think we need to) we want to check that the number of inputs matches what the hints can process, it's up to the hint to perform the check (ie if len(inputs) > ... --> error). Currently this was encapsulated in a Call function.
Similarly, the circuit developer knows how many outputs it expects from a hint. He should set it like so bits := api.NewHint(BitsHint, 220, value) would return the first 220 bits for example in []bits. Previously this was done declaratively at hint definition, through NbOutputs() method, which was called both in the frontend and the backend at solving time. --> it makes no sense as the backend will not decide the number of outputs, the frontend did at compile time and created the exact number of wires needed.

@ivokub did you have specific future scenario in mind when doing this pattern?

NbInputs was already removed from the hint.Function interface before. It was really a leftover in hint.StaticHintFunction construction.
I added separate function for computing the number of output wires for cases when the number of outputs is non-trivial and is computed by some other function. For example, for ternary decomposition the number of outputs is ceil(log_3(modulus)) and it may be error-prone if the user computes it by hand. But, thinking now, we most probably do not want to encourage users to use such complicated hints directly and should rather refer them to use a gadget which provides (correctly) constrained version of them. I'm all good with removing NbOutputs.
In that context - with the creation of std/math / std/math/bits, would it make sense to make the built-in hints internal? In the context of hints considered harmful?


Create 20191228-murata2makoto-JSON-or-XML-in-a-ZIP-package.md

Just a starter.

This is not a good idea in my opinion.  Will increase font loading time & memory consumption significantly for no good benefit; also file size.

OOXML documents are much more compact than binary MS Office documents.  Unless some experiment is carried out, I do not believe that file size will be a problem.  Textual representations have obvious advantages: easier to browse and modify.  The industry has moved away from ASN.1 to XML, JSON, and so forth.

OOXML documents are much more compact than binary MS Office documents. Unless some experiment is carried out, I do not believe that file size will be a problem. Textual representations have obvious advantages: easier to browse and modify. The industry has moved away from ASN.1 to XML, JSON, and so forth.

That's a lot of mixing up things.  Rest of my comments are not addressed either.

Let me ask this way: what problem are you trying to solve?

I just checked your bio.  You should know better that yes, indeed, XML is great for representing "documents" where text is the primary form of content.  Fonts are not such objects though.  Would you use XML to represent images?  Audio/video?

There are already several XML representations of fonts.  I think that this is because binary fonts are harder to browse and modify.  In the area of document formats, textual representations combined with ZIP have overshadowed binary representations.  Why do fonts continue to use binary representations?

I would certainly prefer textual representations for font features and metadata.   Tons of numbers might be different, though.

You are missing the point.
XML is good for authoring format.

At any rate, since no one is willing to make this happen, this discussion is completely moot.

I intend to raise this issue from the JP mirror SC34.  EPUB is a delivery format.  Arguably, OOXML and ODF are used for delivery.

I dislike this.
A delivered font should be simple and straight, and easy to interpret (layout, render, etc.) Therefore, a ZIP with XML inside it is almost the worst option for this purpose. (And that's why I really dislike CFF -- their interpreters are hard to implement.)
XML, JSON and all the general but complex stuff should be only used when designing fonts.

Right.  Fonts need to be random-access.

I’d like to reject this proposal. A more valuable discussion may be creating a textual representation of OpenType fonts — Well, TTX is already there.


vs project

Hi:
can you provide a vs project?
thanks

I need to provide some build documentation.  This project uses the hunter package manager, which is based on CMake.  CMake provides a variety of "generators" that can be used to create build projects for your preferred development environment directly from CMake source, so there are no plans to add explicit projects for each IDE (you can generate them).
Here are some links from that project's documentation:
Hunter docs: https://docs.hunter.sh/en/latest/
Please read "Brief overview": https://docs.hunter.sh/en/latest/overview.html
And try "Quick start": https://docs.hunter.sh/en/latest/quick-start.html
For now, the Appveyor CI tests (see appveyor.yml) can serve as a sort of guide for VS builds until I get a chance to update documentation.  The Travis CI integration (.travis.yml) should cover the non Windows builds.
I make use of the polly project for CMake toolchain abstractions and a small CMake front end called polly.py.  This isn't a hard requirement for builds (you can provide your own toolchains or let CMake use a platform default), but since it is a tested path, it may be easiest to start there.  (This makes the cross platform builds and tests much easier!)
I don't do any development with VS, but the Appveyor tests run regularly, so it should work.  Here is a rough outline of the build process extracted from the root level appveyor.yml.  Note that this will borrow the CI setup, and once you have installed a recent CMake and polly, you won't have to repeat that.  (It may be best to add the environment variables shown to your Windows configuration.)  This is a quick copy and paste w/ reformatting from that file (untested), but it should be close to functional.
set TOOLCHAIN "vs-14-2015-win64-sdk-8-1"
set CONFIG Release

set PATH=C:\Python34-x64;C:\Python34-x64\Scripts;%PATH%
pip install requests

# Install latest Polly toolchains and scripts (replace 'appveyor DownloadFile' w/ you your favorite tool)
appveyor DownloadFile https://github.com/ruslo/polly/archive/master.zip
7z x master.zip
set POLLY_ROOT=%cd%\polly-master

# Install dependencies (CMake, Ninja)
python %POLLY_ROOT%\bin\install-ci-dependencies.py

# Tune locations
set PATH=%cd%\_ci\cmake\bin;%PATH%
set PATH=%cd%\_ci\ninja;%PATH%

# Add '--quiet' to avoid leaking the token to logs
git submodule update --init --recursive --quiet

# MSYS2 location
set MSYS_PATH=C:\msys64\usr\bin

# cd to drishti top level directory and run build
cd drishti
bin\build-appveyor.cmd "%CONFIG%" "%TOOLCHAIN%"

This should build a release with a library and various file based applications.  The real-time facefilter eye tracking demo is focused primarily on iOS and Android devices. The Windows Qt build currently has a small issue https://github.com/elucideye/drishti/issues/179 and is not functional.  I may add a small sfml based real-time demo that would run on Windows in the future.  If you add a --open option to the polly command in build-appveyor.cmd it should open the generated VS project automatically after the first build completes.
python %POLLY_ROOT%\bin\polly.py ^
--open ^
--verbose ^
--archive drishti ^
--config "%1%" ^
--toolchain "%2%" ^
--test ^
--fwd HUNTER_USE_CACHE_SERVERS=ONLY ^
HUNTER_DISABLE_BUILDS=YES ^
DRISHTI_COPY_3RDPARTY_LICENSES=ON ^
DRISHTI_USE_TEXT_ARCHIVES=YES

You can check polly.py --help for a list of all available VS toolchains.  Note that if you use a new toolchain you will have to remove the HUNTER_USE_CACHE_SERVERS and HUNTER_DISABLE_BUILDS options to allow local package builds from source.  Hunter will build any required packages (i.e., OpenCV) along with any transitive dependencies, and a local cache will be populated, so subsequent build will be much faster.  I hope this helps!


added library with all UR robots

This library includes UR3/3e, UR5/53, UR10/10e, UR16e
The meshes were generated from the URDFs in the Universal Robots ROS2 Driver
The DH Parameters are directly from Universal Robots

Hi @madelinegannon, thank you for this PR.
Usually the robot library files are named after the institution/lab/workshop where they are being used.
Would you like to rename them?

@visose I understand the convention, however I made this library so that it’s easier for users to find a specific UR model.
✨Excellent plugin, by the way ✨ — thank you to you and your collaborators for all your hard work.

Ah, sure, a generic library for UR only is good idea. Specially since these are rarely customized and would need tweaking for a particular setup.

Great catch — Thanks for verifying.
Madeline Gannon
atonaton.com    |   @madelinegannon https://twitter.com/madelinegannon
|    1<PHONE_NUMBER>
On Thu, Mar 28, 2024 at 2:12 PM Robin Godwyll @.***>
wrote:

Hey this is a super helpful addition to the plugin.
I did find a small issue though, The DH-Parameters for The UR5e differ
slightly from the ones on the website:
Screenshot.2024-03-28.at.20.07.33.png (view on web)
https://github.com/visose/Robots/assets/25942644/be30c2ff-fd93-4d66-98a4-5b903e49af0b
The difference is small:
162.3 instead of 162.5
-392.25 instead of -392.2
100.3 instead of 99.7
The other robots seem to be fine. Their values correspond with the ones
from UR.
In 99.99% of cases this discrepancy probably doesn't matter, but when
using the plugin in conjunction with other tools that use the Values from
the Website there can be quite a discrepancy in final robot target.
I will create a PR with the updated values.
—
Reply to this email directly, view it on GitHub
https://github.com/visose/Robots/pull/209#issuecomment-2025925413, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAYDRSE3Q73G5PNBB5ULBLDY2RMPTAVCNFSM6AAAAAAT6TMCZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRVHEZDKNBRGM
.
You are receiving this because you were mentioned.Message ID:
@.***>


Add Function.bind polyfill to PDF rendering

Follow on to https://github.com/studentinsights/studentinsights/pull/894#issuecomment-292015698.

selfie

Looks like this did it.


Bit shift right has surprising behavior for Mints

Shifting an integer value right by a large number of bits should yield either 0 or -1 depending on the sign of what is shifted. However, that doesn't happen for Mints. Instead it looks like it goes to the raw machine version of right shift which will perform a shift with the shift value modulo 64 (because it only uses the lower bits).
Example program:
main() {
  // Smi
  var i = 1 << 30;
  var j = -i;
  print(i >> 37);
  print(j >> 37);
  // Mint
  i = 1 << 50;
  j = -i;
  print(i >> 67);
  print(j >> 67);
  // Bigint
  i = i << 120;
  j = -i;
  print(i >> 199);
  print(j >> 199);
}

Should print:
0
-1
0
-1
0
-1
Instead it prints:
0
-1
140737488355328
-140737488355328
0
-1

cc @sgmitrovic @mhausner


auto-detect WAR projects and do the right thing OOTB (also defaulting the webPort to 8080?)

we should be able to default the docker build based on project packaging of war

Yeah I hit that issue today to run an old camel + tomcat project.
I was trying to fiddle with the maven plugin to specify a base docker image (from) and select rootWar and whatnot. But failed misrebly.
I think we need a war generator that creates a docker image that runs in tomcat / jetty (easy to pick which one).

@ro14nd its a bit old school with WAR but it makes a great story for migrating your plain tomcat WARs to docker / kubernetes if all they do is to add the f-m-p.

@jstrachan @davsclaus i have a generator locally built to map to this, will send a PR to soon

awesome!

@jstrachan what are the common file detections you feel we can add to the project to set the  default images ?
some things which I see are

beans.xml/*-ds.xml - can quality for jboss/wildfly
jboss-deployment-structure.xml - can qualify for jboss/wildfly
all others will be fabric8/tomcat-8

@davsclaus https://github.com/kameshsampath/fabric8-maven-plugin/tree/issue-149 is the WIP for webapp(war) generator, the samples/simple-webapp does build a webapp with fabric8/tomcat-8 as base image.. please try it out and let me know if improvements.. am working onto improve it further before a PR

I've totally forgotten the magic file names ;) I thought there was canonical XML files that tomcat/jetty/jboss would look for in /WEB-INF - like jetty-context.xml or something - but can't seem to find them via googling

@jstrachan  - they are META-INF/context.xml for tomcat , and WEB-INF/jboss-deployment-structure.xml or jboss-web.xml for Wildfly

ah cool. for jetty:

WEB-INF/jetty-web.xml e.g.  https://github.com/jetty-project/spring-petclinic-patched/blob/patched/src/main/webapp/WEB-INF/jetty-web.xml
target/jetty-logging.properties e.g. https://github.com/jetty-project/jetty-maven-plugin-examples/blob/master/webapp-2.5/src/main/resources/jetty-logging.properties

@jstrachan right now the application should detect the image based magic files we discussed so far.

As we have a webapp generator now, this issue can be closed.


Fix issue with copy shader not working in multiview

This should fix this error reported by @Malcolmnixon :

In multiview uv_interp is a vec3 variable, we need to swizzle it to allow this multiplication.

@Malcolmnixon if you can test this and verify it indeed fixes your issue? I didn't reproduce the issue but in theory this should fix it.

@clayjohn I'll let you evaluate whether we'd like to cherry-pick both or neither.

Let's cherrypick both!

Thanks!


GUACAMOLE-39: Do not abort VNC/RDP connection if SFTP fails

Re-implementation in the current code base of the changes in the previous pull request for GUACAMOLE-39.  For VNC/RDP connections, this PR now logs the failure rather than aborting the connection.  This allows the primary connection to succeed regardless of the secondary SFTP connection.

In theory it was helpful for a feature in our product but said feature turned out not so well.  This change also exposes a problem similar to https://issues.apache.org/jira/browse/GUACAMOLE-257.  It may let you in without problems but do not try to use drag and drop, it will kick you out and there's a segfault in the logs.  At this point we do not have the resources to make this work as well as it should. So we're abandoning this change.


Handling of expired tokens is not great

If a user is logged in for a long time and the token expires or is revoked, it's not clear that they need to log in again. PDlive should support token refresh and should take the user to the login page if the token is invalid and can't be refreshed.

Not planned; project deprecated in favour of Operations Console


Improvements to cache keys and time multiplier constants

Addresses review points from https://github.com/sunspikes/php-ratelimiter/pull/5

Thank you @Feijs


Restructure docs directory

Description of issue or feature request:
Current behavior:
No clear distinction of different documentation types in docs directory.
Expected behavior:
Clear distinction of different documentation types in docs directory, e.g. along the following categories:

adr
rtd (e.g. see sphinx-quickstart dir scaffolding)
misc docs that are neither an adr, nor rendered on rtd (?)

related: see #1848

Related issue: https://github.com/theupdateframework/python-tuf/pull/1853#pullrequestreview-878492917

For background, this is confusing because when RTD docs were created, while the new reference docs were added in a separate docs/api/  we wanted to use the existing docs in RTD without removing them from easily findable locations in git...
Like the previous link shows this approach does have some problems and maybe it makes sense draw a more strict line between "user documentation"  (api reference, installation, examples published on RTD) and internal developer documentation (only in git). It's not always so clear cut though: things like CONTRIBUTORS are edge cases.

Example from pyca/cryptography. In this case they have a

short GitHub-render-friendly CONTRIBUTING.rst in the canonical location, which just refers to the detailed
development documentation on RTD


OfflinePlugin: ServiceWorker entry has more than one output file, only first will be used

I'm using webpack dev server, and to use inline mode I need to include the dev server entry in my webpack configuration.
var config = require("./webpack.config.js");
config.entry.app.unshift("webpack-dev-server/client?http://localhost:8080");
var compiler = webpack(config);
var server = new webpackDevServer(compiler, {...});
server.listen(8080);

This effectively leaves my webpack with:
{
    entry: {
        app: [
            "webpack-dev-server/client?http://localhost:8080",
            "./app.js"
        ]
    }

With the warning that OfflinePlugin produces, I can't figure out what problem it causes or what I can do to get rid of the warning other than just not use Webpack's dev server (not an option).

Thanks for the report, @AsaAyers
I personally do not use webpack-dev-server, so there is a good change that offline-plugin may not work with it.
About this particular warning, it's a bit complicated and in ideal world it should not happen. I thought it won't happen ever. It works this way:
To correctly work, ServiceWorker creates fake entryю Every entry is actually a chunk, and chunks in webpack by some reason has array files instead of one file. So it's possible that chunk may have more than one files and looking through webpack's code, it seems that it used only for hot-reload. Probably some kind of hack. This is why that warning exists, I simply do not understand why there might be more than one files in output for one chunk, plus it shouldn't modified anyway for ServiceWorker's fake entry. So, in case of  hot reloading, it probably could be simply ignored since hot reloading shouldn't be embedded/used inside ServiceWorker.
Anyway, I have in plans to improve offline-plugin work with hot reloading, probably just disabling it in such mode. Btw, why you may need it with hot reloading? It's actually conflicting things since hot reloading is intended to load fresh files from server as soon as they changed, but offline-plugin is intended to load them from cache.

I'm not using hot reloading. I just have it setup so that when any files change it tells the browser to refresh and load the new version. It seems like that should be fine if offline-plugin is using hash mode, I'm not sure about the others.

Does it need to be a 2nd entry in a chunk? Could the service worker just be it's own chunk?

Does it need to be a 2nd entry in a chunk? Could the service worker just be it's own chunk?

SW has separate entry/chunk. It should be first and is first in files array. But plugin expects only it to there, so when there is more things (some other plugin added files) you get a warning about it, warning means that something may not work as expected since unexpected thing happened. However, it's not an error, just a warning, this means that SW entry was successfully compiled.

I'm not using hot reloading. I just have it setup so that when any files change it tells the browser to refresh and load the new version. It seems like that should be fine if offline-plugin is using hash mode, I'm not sure about the others.

Should be problem in any more I think. To update cache, SW anyway needs to be reloaded or update function needs to be called. So on each reload, you need one more reload to update cache. This could work, but what is the reason then to use it in dev mode?
I actually do not want to use plugin dev mode myself since I had to reload page twice to get update version. Problem is that at them moment runtime.js is a template and fails with systax error if plugin is not included, which I see as a right way to develop with offline-plugin: develop with it disabled, then before releasing enable it and test offline support.
Here is issue for it https://github.com/NekR/offline-plugin/issues/4 but I had no time yet for it yet. Would be good to fix anyway.

Okay, fixed in 2.0.0. Should work normal with Webpack Dev Server. No with Hot Reloading or Live Reloading of course. With lazy mode it won't work too. Those modes are simply incompatible with this plugin and there is no sense to support them (or probably no way).
To test if everything works fine with ServiceWorker and AppCache -- just run regular server (or webpack server with simple options) for your dev/prod envs. You do not need to do it offset since almost everything is handled by offline-plugin for you.


Trying one more time to improve closures...

Trying again #8588 to see if it crashes on linux. If so, I'll try to debug it and fix it. The code and the names used in the code are temporary, I'll improve them.

Oh, nevermind... I just don't have time.


Should use <IP_ADDRESS> in agent mbus url for director

For addressing tracker #133906403,
@maximilien  Please review this PR after PR 216 is merged, thanks.

@maximilien Rebased, please review, thanks.

@maximilien corrected some typos, it's urgent for review and merge. Thanks.

Hi @maximilien, I added some UT code to verify if mbus in agentOptions is as expected. Since agentOptions is a private variable, I have to add a new public function to retrieve the value of agentOptions. Please review. Thanks.

OK, see my comment about VcapPassword now being visible and needing to be obfuscated? See some of my suggestions in the comment.

Hi @maximilien, it's strange that I can't find your comment about VcapPassword, could you please post it as a general comment here again? Thanks.

My main concern here is that now the VcapPassword which is pretty serious information is now accessible in all other packages and could potentially be made public in future PRs or when logging etc.
I wonder if we could avoid making the entire AgentOptions public? or does VcapPassword need to be public? or perhaps when returning return c.agentOptions here you simply set the VcapPassword to ***** in the copied structure?

See ^^^

OK what's the status of this. Depending PR is now merged.

@maximilien I changed vcap password to a private variable, and use public Get/Set functions to manipulate it. Please help review my changes, thanks.

@maximilien I modified the code and please review. Thanks.

@mattcui ,
OK merging. But I really would like @zhanggbj and @gu-bin to also double check this in pipeline, making sure all is fine.
FYI that increasingly I would like you all to peer-review each other's code. I will request that, make sure you do so and confirm pipeline as well.
Best,
Max


Resolve #219 - Handle t1__ more objects

Builds on @czf 's changes in PR 224 to allow for More in Comment response.
Resolves #219 and now handles "Continue this thread" More-like objects eg:
{
  "count": 0,
  "name": "t1__",
  "id": "_",
  "parent_id": "t1_cydnx3d",
  "depth": 10,
  "children": []
}

I'd like a couple changes on that, but otherwise looks good. Thanks!

@Slord6 is there a test you can add in addition to GetCommentsWithMoresAsync or modify that test to cover the missed case?


Use less memory during lora state dict validation

Context
This partially addresses #665, to fully address it we also need to support QLoRA for Mistral 7B.
Now that we've enabled QLoRA with fp32 compute dtype, we can support fine-tunes on T4. The only remaining blocker is our LoRA state dict validation, which uses extra memory because of its calls to .state_dict().keys() for validation. For single device, we can actually just use the missing and unexpected keys as returned by .load_state_dict(strict=False). (The only reason we wrote the validate_state_dict_for_lora function to begin with was because FSDP's missing and unexpected keys still contained FSDP-specific substrings, though this is fixed in the nightlies after https://github.com/pytorch/pytorch/pull/120600).
Changelog
This PR adds an equivalent method to validate_state_dict_for_lora but without the additional .state_dict() call. It integrates this function into lora_finetune_single_device (but not lora_finetune_distributed, as otherwise we would need to constrain to PyTorch nightlies only).
Test plan
Added a unit test
pytest tests/modules/peft/test_peft_utils.py
...
====== 24 passed, 1 warning in 0.14s ==========

tune run lora_finetune_single_device --config llama2/7B_lora_single_device checkpointer.checkpoint_dir=/data/users/ebs/checkpoints checkpointer.checkpoint_files=['llama2-7b-torchtune.pt'] checkpointer=torchtune.utils.FullModelTorchTuneCheckpointer tokenizer.path=/data/users/ebs/checkpoints/lora-debug/tokenizer.model
...
1|178|Loss: 0.9532400369644165:   1%|▉

After making the changes in this PR on Colab:

Hey Team, @ebsmothers
Now I am getting the below. :
RuntimeError: bf16 precision was requested but not available on this hardware. Please use fp32 precision instead.
Here is my command :
!tune run recipes/lora_finetune_single_device.py --config /content/torchtune/recipes/configs/mistral/7B_lora_single_device.yaml
batch_size: 4
checkpointer:
component: torchtune.utils.FullModelHFCheckpointer
checkpoint_dir: /tmp/Mistral-7B-v0.1
checkpoint_files:

pytorch_model-00001-of-00002.bin
pytorch_model-00002-of-00002.bin
model_type: MISTRAL
output_dir: /tmp/Mistral-7B-v0.1
recipe_checkpoint: null
compile: false
dataset:
component: torchtune.datasets.alpaca_dataset
train_on_input: true
device: cuda
dtype: bf16
enable_activation_checkpointing: true
epochs: 3
gradient_accumulation_steps: 4
log_every_n_steps: null
loss:
component: torch.nn.CrossEntropyLoss
lr_scheduler:
component: torchtune.modules.get_cosine_schedule_with_warmup
num_warmup_steps: 100
max_steps_per_epoch: null
metric_logger:
component: torchtune.utils.metric_logging.DiskLogger
log_dir: /tmp/Mistral-7B-v0.1
model:
component: torchtune.models.mistral.lora_mistral_7b
apply_lora_to_mlp: true
apply_lora_to_output: true
lora_alpha: 16
lora_attn_modules:
q_proj
k_proj
v_proj
lora_rank: 64
optimizer:
component: torch.optim.AdamW
lr: 2.0e-05
output_dir: /tmp/Mistral-7B-v0.1
profiler:
component: torchtune.utils.profiler
enabled: false
output_dir: /tmp/alpaca-llama2-finetune/torchtune_perf_tracing.json
resume_from_checkpoint: false
seed: null
shuffle: true
tokenizer:
component: torchtune.models.mistral.mistral_tokenizer
path: /tmp/Mistral-7B-v0.1/tokenizer.model

Traceback (most recent call last):
File "/usr/local/bin/tune", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.10/dist-packages/torchtune/_cli/tune.py", line 49, in main
parser.run(args)
File "/usr/local/lib/python3.10/dist-packages/torchtune/_cli/tune.py", line 43, in run
args.func(args)
File "/usr/local/lib/python3.10/dist-packages/torchtune/_cli/run.py", line 179, in _run_cmd
self._run_single_device(args)
File "/usr/local/lib/python3.10/dist-packages/torchtune/_cli/run.py", line 93, in _run_single_device
runpy.run_path(str(args.recipe), run_name="main")
File "/usr/lib/python3.10/runpy.py", line 289, in run_path
return _run_module_code(code, init_globals, run_name,
File "/usr/lib/python3.10/runpy.py", line 96, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "recipes/lora_finetune_single_device.py", line 509, in
sys.exit(recipe_main())
File "/usr/local/lib/python3.10/dist-packages/torchtune/config/_parse.py", line 50, in wrapper
sys.exit(recipe_main(conf))
File "recipes/lora_finetune_single_device.py", line 502, in recipe_main
recipe = LoRAFinetuneRecipeSingleDevice(cfg=cfg)
File "recipes/lora_finetune_single_device.py", line 99, in init
self._dtype = utils.get_dtype(cfg.dtype, device=self._device)
File "/usr/local/lib/python3.10/dist-packages/torchtune/utils/precision.py", line 119, in get_dtype
raise RuntimeError(
RuntimeError: bf16 precision was requested but not available on this hardware. Please use fp32 precision instead.

This is for the free T4 google colab


manage content sync

Requires: https://github.com/ansible/galaxy_ng/pull/400
Finishes off the work on https://github.com/ansible/ansible-hub-ui/pull/164

LGTM :+1:


Ubuntu 编译helloworld@esp8266烧录报错

Describe the bug

编译成功
烧录报错

> Executing task: aos upload helloworld@developerkit <

aos-cube version: 0.2.60
[INFO]: Target: helloworld@developerkit
[INFO]: Currently in aos_sdk_path: '/home/xiaodi/work/AliOS-Things'
SCons import failed. Unable to find engine files in:
  /usr/local/bin/../engine
  /usr/local/bin/scons-local-3.0.1
  /usr/local/bin/scons-local
  /usr/local/lib/python2.7/dist-packages/lib/scons-3.0.1
  /usr/local/lib/scons-3.0.1
  /usr/lib/scons-3.0.1
  /usr/local/lib/scons-3.0.1
  /usr/local/lib/python2.7/dist-packages/lib/python2.7/site-packages/scons-3.0.1
  /usr/local/lib/python2.7/site-packages/scons-3.0.1
  /usr/lib/python2.7/site-packages/scons-3.0.1
  /usr/local/lib/python2.7/site-packages/scons-3.0.1
  /usr/lib/scons-3.0.1
  /usr/local/lib/python2.7/dist-packages/lib/scons
  /usr/local/lib/scons
  /usr/lib/scons
  /usr/local/lib/scons
  /usr/local/lib/python2.7/dist-packages/lib/python2.7/site-packages/scons
  /usr/local/lib/python2.7/site-packages/scons
  /usr/lib/python2.7/site-packages/scons
  /usr/local/lib/python2.7/site-packages/scons
  /usr/lib/scons
Traceback (most recent call last):
  File "/usr/local/bin/scons", line 192, in <module>
    import SCons.Script
ImportError: No module named SCons.Script
[AliOS-Things] ERROR: "s" returned error code 1.


Feature: Specific help messages for each subcommand

As mentioned in #174.

Implemented in 1.9.0-beta


Please use @data-path rather than @path when including files

When bikeshed includes an external file, it adds a path attribute at the point of inclusion, which is invalid HTML and has to be fixed manually when publishing to /TR

Attribute “path” not allowed on element “pre” at this point.

Changing this to data-path would solve this (which is what I did manually)

That's specifically on a code inclusion, right? An element with class="include-code"?
If so, I fixed that earlier this week; it's in the latest version. ^_^


Adds initial GitHub Actions CI - testing w/ no name

work includes:

First CI yaml! :tada:

LGTM!


Dont log anything

Hy,
probably i misunderstand something... , i cant make logs with loglevel...
// same as when use var log = require('loglevel');
import * as log from 'loglevel';
//....

render(){
    log.info("........................im here");
    log.debug("all done");
    //...
}

"dependencies": {
    "ajv": "^5.5.0",
    "ajv-errors": "^1.0.0",
    "auth0-js": "^8.11.3",
    "localforage": "^1.5.3",
    "loglevel": "^1.6.0",
    "prop-types": "^15.6.0",
    "react": "^16.1.1",
    "react-dom": "^16.1.1",
    "react-redux": "^5.0.6",
    "react-router": "^4.2.0",
    "react-router-dom": "^4.2.2",
    "react-scripts": "^1.0.17",
//...
  }

nothing logged out, no errors...

The default loglevel is warn, so info and debug messages are hidden. Try log.setLevel('warn'), or log.warn('a message'). Does that work for you?

Sorry, that should have been log.setLevel('debug')

thats intersting. i changed as you suggest , its working:
import * as log from 'loglevel';
log.setLevel('debug')
//...
log.info("........................im here");
log.warn("all done");

still working when remove  log.setLevel('debug'), restart environment...
thank you for quick response :)

Yes, that will still be working after you remove that line because by default loglevel persists your set level in your browser local storage (or cookies, if not available). If you clear your cookies or try incognito you'll go back to warn level, and these messages will disappear again.
I'd recommend setting a sensible default level in your code (try log.setDefaultLevel('something')) for whatever you're level you'd like to be visible normally, and then using log.setLevel yourself by hand when you need to make it louder/quieter for some specific debugging.
Anyway, glad your issue is sorted, let me know if you have any other trouble.


Including time in the date picker?

Is it possible to include the current time in the date picker. I know it is a datepicker but having a time option would be great also.

+1
Came here looking for an issue already opened or to open my own for this exact reason. Seems there are a number of datepickers out there (this one being my favorite), and only a limited number of time pickers.

Thanks for you interest in the time picking feature. We currently don't support time picking and unfortunately due to limited time and resources I don't see this feature being added any time soon.
If you really need an all in one datetimepicker you should check Eonasdan/bootstrap-datetimepicker