[QUESTION] Infinite Campus security on User Data xml file

I know this project is long outdated but I was wondering if anyone can help me with a problem I am having. I am rewriting the code into php, and I've gotten to the point where the code is attempting to get the file content of a page, given district app name and code. When you type the url on the tab directly, it display all the user information. But when you use file_get_content of the same file, it has a security block on it.
Help would be appreciated.

You may have to spoof a user-agent for it to work (like the one in your web
browser’s request headers)
On Mon, Feb 17, 2020 at 11:33, jbanaqa<EMAIL_ADDRESS>wrote:

I know this project is long outdated but I was wondering if anyone can
help me with a problem I am having. I am rewriting the code into php, and
I've gotten to the point where the code is attempting to get the file
content of a page, given district app name and code. When you type the url
on the tab directly, it display all the user information. But when you use
file_get_content of the same file, it has a security block on it.
Help would be appreciated.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/shinyquagsire23/InfiniteCampusAPI/issues/8?email_source=notifications&email_token=ABDXBHXLHVVYFPIQBV5L5WTRDLRCDA5CNFSM4KWXMJE2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IOD3IOA,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABDXBHTVPSNPH2JUHEVZZOTRDLRCDANCNFSM4KWXMJEQ
.

--
Ari Stassinopoulos
https://ari-s.com/

You may have to spoof a user-agent for it to work (like the one in your web browser’s request headers)
On Mon, Feb 17, 2020 at 11:33, jbanaqa @.***> wrote: I know this project is long outdated but I was wondering if anyone can help me with a problem I am having. I am rewriting the code into php, and I've gotten to the point where the code is attempting to get the file content of a page, given district app name and code. When you type the url on the tab directly, it display all the user information. But when you use file_get_content of the same file, it has a security block on it. Help would be appreciated. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#8?email_source=notifications&email_token=ABDXBHXLHVVYFPIQBV5L5WTRDLRCDA5CNFSM4KWXMJE2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IOD3IOA>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDXBHTVPSNPH2JUHEVZZOTRDLRCDANCNFSM4KWXMJEQ .
-- Ari Stassinopoulos https://ari-s.com/

Thank you for your help. I'm sorry as I'm very entry level with API's and coding in general. Can you elaborate on your advice?
Thanks.

You may have to spoof a user-agent for it to work (like the one in your web browser’s request headers)
On Mon, Feb 17, 2020 at 11:33, jbanaqa @.***> wrote: I know this project is long outdated but I was wondering if anyone can help me with a problem I am having. I am rewriting the code into php, and I've gotten to the point where the code is attempting to get the file content of a page, given district app name and code. When you type the url on the tab directly, it display all the user information. But when you use file_get_content of the same file, it has a security block on it. Help would be appreciated. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#8?email_source=notifications&email_token=ABDXBHXLHVVYFPIQBV5L5WTRDLRCDA5CNFSM4KWXMJE2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IOD3IOA>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDXBHTVPSNPH2JUHEVZZOTRDLRCDANCNFSM4KWXMJEQ .
-- Ari Stassinopoulos https://ari-s.com/

Thank you for your help. I'm sorry as I'm very entry level with API's and coding in general. Can you elaborate on your advice?
Thanks.

Infinite Campus might be blocking requests that don't have a proper user-agent header, which is used to describe where a request is coming from basically. You can google the user-agent used by browsers such as Chrome, Safari or Firefox and set your own user-agent to the exact same thing so your request will probably get through.
Also if you could elaborate on security block, do you mean a 403 or a specific infinite campus security incident scare page? I've only seen the ladder when I was trying to access data of other students without sending a proper auth request as them.

You may have to spoof a user-agent for it to work (like the one in your web browser’s request headers)
On Mon, Feb 17, 2020 at 11:33, jbanaqa @.***> wrote: I know this project is long outdated but I was wondering if anyone can help me with a problem I am having. I am rewriting the code into php, and I've gotten to the point where the code is attempting to get the file content of a page, given district app name and code. When you type the url on the tab directly, it display all the user information. But when you use file_get_content of the same file, it has a security block on it. Help would be appreciated. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#8?email_source=notifications&email_token=ABDXBHXLHVVYFPIQBV5L5WTRDLRCDA5CNFSM4KWXMJE2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IOD3IOA>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDXBHTVPSNPH2JUHEVZZOTRDLRCDANCNFSM4KWXMJEQ .
-- Ari Stassinopoulos https://ari-s.com/

Thank you for your help. I'm sorry as I'm very entry level with API's and coding in general. Can you elaborate on your advice?
Thanks.

Infinite Campus might be blocking requests that don't have a proper user-agent header, which is used to describe where a request is coming from basically. You can google the user-agent used by browsers such as Chrome, Safari or Firefox and set your own user-agent to the exact same thing so your request will probably get through.
Also if you could elaborate on security block, do you mean a 403 or a specific infinite campus security incident scare page? I've only seen the ladder when I was trying to access data of other students without sending a proper auth request as them.

Thank you for your response. The data that is being pulled from the file_get_content is one string: "Contact your school or district's tech staff to receive the proper url to enter your username and password."
This data is only shown when pulled through the php function, for when I load in the same URL, it returns the data I'm looking for.

You may have to spoof a user-agent for it to work (like the one in your web browser’s request headers)
On Mon, Feb 17, 2020 at 11:33, jbanaqa @.***> wrote: I know this project is long outdated but I was wondering if anyone can help me with a problem I am having. I am rewriting the code into php, and I've gotten to the point where the code is attempting to get the file content of a page, given district app name and code. When you type the url on the tab directly, it display all the user information. But when you use file_get_content of the same file, it has a security block on it. Help would be appreciated. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#8?email_source=notifications&email_token=ABDXBHXLHVVYFPIQBV5L5WTRDLRCDA5CNFSM4KWXMJE2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IOD3IOA>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDXBHTVPSNPH2JUHEVZZOTRDLRCDANCNFSM4KWXMJEQ .
-- Ari Stassinopoulos https://ari-s.com/

Thank you for your help. I'm sorry as I'm very entry level with API's and coding in general. Can you elaborate on your advice?
Thanks.

Assuming that the request goes through and your php.ini allows remote requests, you want something like the following (adapted from the PHP docs)
$opts = array(
  'http'=>array(
    'method'=>"GET",
    'header'=>"User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36")
);

$context = stream_context_create($opts);

// Open the file using the HTTP headers set above
$file = file_get_contents('http://URL/...', false, $context);

You may have to spoof a user-agent for it to work (like the one in your web browser’s request headers)
On Mon, Feb 17, 2020 at 11:33, jbanaqa @.***> wrote: I know this project is long outdated but I was wondering if anyone can help me with a problem I am having. I am rewriting the code into php, and I've gotten to the point where the code is attempting to get the file content of a page, given district app name and code. When you type the url on the tab directly, it display all the user information. But when you use file_get_content of the same file, it has a security block on it. Help would be appreciated. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#8?email_source=notifications&email_token=ABDXBHXLHVVYFPIQBV5L5WTRDLRCDA5CNFSM4KWXMJE2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IOD3IOA>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDXBHTVPSNPH2JUHEVZZOTRDLRCDANCNFSM4KWXMJEQ .
-- Ari Stassinopoulos https://ari-s.com/

Thank you for your help. I'm sorry as I'm very entry level with API's and coding in general. Can you elaborate on your advice?
Thanks.

Assuming that the request goes through and your php.ini allows remote requests, you want something like the following (adapted from the PHP docs)
$opts = array(
  'http'=>array(
    'method'=>"GET",
    'header'=>"User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36")
);

$context = stream_context_create($opts);

// Open the file using the HTTP headers set above
$file = file_get_contents('http://URL/...', false, $context);

The data returned using the code above is still "Contact your school or district's tech staff to receive the proper url to enter your username and password."
Perhaps the requests aren't going through?

Which page is it that you are trying to access?
On Mon, Feb 17, 2020 at 6:08 PM jbanaqa<EMAIL_ADDRESS>wrote:

You may have to spoof a user-agent for it to work (like the one in your
web browser’s request headers)
On Mon, Feb 17, 2020 at 11:33, jbanaqa @.***> wrote: I know this
project is long outdated but I was wondering if anyone can help me with a
problem I am having. I am rewriting the code into php, and I've gotten to
the point where the code is attempting to get the file content of a page,
given district app name and code. When you type the url on the tab
directly, it display all the user information. But when you use
file_get_content of the same file, it has a security block on it. Help
would be appreciated. — You are receiving this because you are subscribed
to this thread. Reply to this email directly, view it on GitHub <#8
https://github.com/shinyquagsire23/InfiniteCampusAPI/issues/8?email_source=notifications&email_token=ABDXBHXLHVVYFPIQBV5L5WTRDLRCDA5CNFSM4KWXMJE2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IOD3IOA>,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABDXBHTVPSNPH2JUHEVZZOTRDLRCDANCNFSM4KWXMJEQ
.
-- Ari Stassinopoulos https://ari-s.com/
Thank you for your help. I'm sorry as I'm very entry level with API's and
coding in general. Can you elaborate on your advice?
Thanks.
Assuming that the request goes through and your php.ini allows remote
requests, you want something like the following (adapted from the PHP docs
https://www.php.net/manual/en/function.file-get-contents.php)
$opts = array(
'http'=>array(
'method'=>"GET",

'header'=>"User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36")

);
$context = stream_context_create($opts);
// Open the file using the HTTP headers set above
$file = file_get_contents('http://URL/...', false, $context);
The data returned using the code above is still "Contact your school or
district's tech staff to receive the proper url to enter your username and
password."
Perhaps the requests aren't going through?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/shinyquagsire23/InfiniteCampusAPI/issues/8?email_source=notifications&email_token=AKD24J43CM2WVQG55FUSXJTRDM7JHA5CNFSM4KWXMJE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMAI63Y#issuecomment-587239279,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKD24J3W4GDW3Q65MB3CWY3RDM7JHANCNFSM4KWXMJEQ
.

--
Ari Stassinopoulos
https://ari-s.com/

Which page is it that you are trying to access?
…
On Mon, Feb 17, 2020 at 6:08 PM jbanaqa @.> wrote: You may have to spoof a user-agent for it to work (like the one in your web browser’s request headers) On Mon, Feb 17, 2020 at 11:33, jbanaqa @.> wrote: I know this project is long outdated but I was wondering if anyone can help me with a problem I am having. I am rewriting the code into php, and I've gotten to the point where the code is attempting to get the file content of a page, given district app name and code. When you type the url on the tab directly, it display all the user information. But when you use file_get_content of the same file, it has a security block on it. Help would be appreciated. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#8 <#8>?email_source=notifications&email_token=ABDXBHXLHVVYFPIQBV5L5WTRDLRCDA5CNFSM4KWXMJE2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IOD3IOA>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDXBHTVPSNPH2JUHEVZZOTRDLRCDANCNFSM4KWXMJEQ . -- Ari Stassinopoulos https://ari-s.com/ Thank you for your help. I'm sorry as I'm very entry level with API's and coding in general. Can you elaborate on your advice? Thanks. Assuming that the request goes through and your php.ini allows remote requests, you want something like the following (adapted from the PHP docs https://www.php.net/manual/en/function.file-get-contents.php) $opts = array( 'http'=>array( 'method'=>"GET", 'header'=>"User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36") ); $context = stream_context_create($opts); // Open the file using the HTTP headers set above $file = file_get_contents('http://URL/...', false, $context); The data returned using the code above is still "Contact your school or district's tech staff to receive the proper url to enter your username and password." Perhaps the requests aren't going through? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8?email_source=notifications&email_token=AKD24J43CM2WVQG55FUSXJTRDM7JHA5CNFSM4KWXMJE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMAI63Y#issuecomment-587239279>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKD24J3W4GDW3Q65MB3CWY3RDM7JHANCNFSM4KWXMJEQ .
-- Ari Stassinopoulos https://ari-s.com/

Given the district base URL as well as the app name:
(District URL) prism?x=portal.PortalOutline&appName=(District App Name)
When I place the URL in manually, I am able to see the XML file with all the user data, but it isn't being returned with the file_get_content function.

Have you gone through the log-in process? If not you will have to do that
(and make sure the cookies are stored).
On Mon, Feb 17, 2020 at 6:14 PM jbanaqa<EMAIL_ADDRESS>wrote:

Which page is it that you are trying to access?
… <#m_-7971168617543240338_>
On Mon, Feb 17, 2020 at 6:08 PM jbanaqa @.> wrote: You may have to
spoof a user-agent for it to work (like the one in your web browser’s
request headers) On Mon, Feb 17, 2020 at 11:33, jbanaqa @.> wrote: I
know this project is long outdated but I was wondering if anyone can help
me with a problem I am having. I am rewriting the code into php, and I've
gotten to the point where the code is attempting to get the file content of
a page, given district app name and code. When you type the url on the tab
directly, it display all the user information. But when you use
file_get_content of the same file, it has a security block on it. Help
would be appreciated. — You are receiving this because you are subscribed
to this thread. Reply to this email directly, view it on GitHub <#8
https://github.com/shinyquagsire23/InfiniteCampusAPI/issues/8 <#8
https://github.com/shinyquagsire23/InfiniteCampusAPI/issues/8>?email_source=notifications&email_token=ABDXBHXLHVVYFPIQBV5L5WTRDLRCDA5CNFSM4KWXMJE2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IOD3IOA>,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABDXBHTVPSNPH2JUHEVZZOTRDLRCDANCNFSM4KWXMJEQ
. -- Ari Stassinopoulos https://ari-s.com/ Thank you for your help. I'm
sorry as I'm very entry level with API's and coding in general. Can you
elaborate on your advice? Thanks. Assuming that the request goes through
and your php.ini allows remote requests, you want something like the
following (adapted from the PHP docs
https://www.php.net/manual/en/function.file-get-contents.php) $opts =
array( 'http'=>array( 'method'=>"GET", 'header'=>"User-Agent: Mozilla/5.0
(Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/58.0.3029.110 Safari/537.36") ); $context =
stream_context_create($opts); // Open the file using the HTTP headers set
above $file = file_get_contents('http://URL/...', false, $context); The
data returned using the code above is still "Contact your school or
district's tech staff to receive the proper url to enter your username and
password." Perhaps the requests aren't going through? — You are
receiving this because you commented. Reply to this email directly, view it
on GitHub <#8
https://github.com/shinyquagsire23/InfiniteCampusAPI/issues/8?email_source=notifications&email_token=AKD24J43CM2WVQG55FUSXJTRDM7JHA5CNFSM4KWXMJE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMAI63Y#issuecomment-587239279>,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKD24J3W4GDW3Q65MB3CWY3RDM7JHANCNFSM4KWXMJEQ
.
-- Ari Stassinopoulos https://ari-s.com/
Given the district base URL as well as the app name:
(District URL) prism?x=portal.PortalOutline&appName=(District App Name)
When I place the URL in manually, I am able to see the XML file with all
the user data, but it isn't being returned with the file_get_content
function.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/shinyquagsire23/InfiniteCampusAPI/issues/8?email_source=notifications&email_token=ABDXBHVX22F5MGJACCZH7RTRDNABXA5CNFSM4KWXMJE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMAJJTY#issuecomment-587240655,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABDXBHQ65ZZAWZB67OA3XTDRDNABXANCNFSM4KWXMJEQ
.

--
Ari Stassinopoulos
https://ari-s.com/

Based on the error message, it sounds like the request is unauthenticated.
The reason it works in the browser is because you've previously gone through the login page, and the Infinite Campus server has sent back a login that your browser stores and sends on the next request. PHP doesn't have the browser's cookies, so when it makes the request, the Infinite Campus server doesn't see any login cookie, so effectively you're not logged in.
You need to implement attemptLogin, and get a login cookie before you try to get the info.

Thank you all for the help, I wasn't expecting all the support from an old project.
I realized I was blind from seeing how the XML file only loaded when I logged in previously, and I will implement the login cookies into my server.
Thanks for all the help!

Based on the error message, it sounds like the request is unauthenticated.
The reason it works in the browser is because you've previously gone through the login page, and the Infinite Campus server has sent back a login that your browser stores and sends on the next request. PHP doesn't have the browser's cookies, so when it makes the request, the Infinite Campus server doesn't see any login cookie, so effectively you're not logged in.
You need to implement attemptLogin, and get a login cookie before you try to get the info.

Thank you for your response.
Can you steer me into the right direction for rewriting the implementation of login cookies from this project's java version to php?

Have you gone through the log-in process? If not you will have to do that (and make sure the cookies are stored).
…
On Mon, Feb 17, 2020 at 6:14 PM jbanaqa @.*> wrote: Which page is it that you are trying to access? … <#m_-7971168617543240338_> On Mon, Feb 17, 2020 at 6:08 PM jbanaqa @.> wrote: You may have to spoof a user-agent for it to work (like the one in your web browser’s request headers) On Mon, Feb 17, 2020 at 11:33, jbanaqa @.> wrote: I know this project is long outdated but I was wondering if anyone can help me with a problem I am having. I am rewriting the code into php, and I've gotten to the point where the code is attempting to get the file content of a page, given district app name and code. When you type the url on the tab directly, it display all the user information. But when you use file_get_content of the same file, it has a security block on it. Help would be appreciated. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#8 <#8> <#8 <#8>>?email_source=notifications&email_token=ABDXBHXLHVVYFPIQBV5L5WTRDLRCDA5CNFSM4KWXMJE2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IOD3IOA>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDXBHTVPSNPH2JUHEVZZOTRDLRCDANCNFSM4KWXMJEQ . -- Ari Stassinopoulos https://ari-s.com/ Thank you for your help. I'm sorry as I'm very entry level with API's and coding in general. Can you elaborate on your advice? Thanks. Assuming that the request goes through and your php.ini allows remote requests, you want something like the following (adapted from the PHP docs https://www.php.net/manual/en/function.file-get-contents.php) $opts = array( 'http'=>array( 'method'=>"GET", 'header'=>"User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36") ); $context = stream_context_create($opts); // Open the file using the HTTP headers set above $file = file_get_contents('http://URL/...', false, $context); The data returned using the code above is still "Contact your school or district's tech staff to receive the proper url to enter your username and password." Perhaps the requests aren't going through? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8 <#8>?email_source=notifications&email_token=AKD24J43CM2WVQG55FUSXJTRDM7JHA5CNFSM4KWXMJE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMAI63Y#issuecomment-587239279>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKD24J3W4GDW3Q65MB3CWY3RDM7JHANCNFSM4KWXMJEQ . -- Ari Stassinopoulos https://ari-s.com/ Given the district base URL as well as the app name: (District URL) prism?x=portal.PortalOutline&appName=(District App Name) When I place the URL in manually, I am able to see the XML file with all the user data, but it isn't being returned with the file_get_content function. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8?email_source=notifications&email_token=ABDXBHVX22F5MGJACCZH7RTRDNABXA5CNFSM4KWXMJE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMAJJTY#issuecomment-587240655>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDXBHQ65ZZAWZB67OA3XTDRDNABXANCNFSM4KWXMJEQ .
-- Ari Stassinopoulos https://ari-s.com/

Do the login cookies need to match identically to what Infinite Campus is expecting? If so, where can I find what they're looking for so I can emulate those cookies?

The 'Set-Cookie' HTTP response header is sent by the server; it contains a string and some metadata that the browser parses, and then the browser will send it back when it makes its next request to that site, putting the string in the 'Cookie' HTTP request header.
The Java code con.getHeaderFields().get("Set-Cookie"); is getting a list of all cookies set by the server. The code after that is parsing each one to get rid of everything after the first semicolon. And it appends all cookies being returned together with semicolons, since that's the format for sending multiple cookies in the next request. The con.setRequestProperty("Cookie", cookies); code is what actually sends the cookie in the next request.
I haven't used PHP much, so I'm not sure what the best way of doing things is, but from some basic searching, it seems like there's an $httpresponseheaders variable that gets populated when you do file_get_contents() (documentation). You should be able to check that array for any entries that start with 'Set-Cookie', and do the same parsing that the Java version does.
For sending back the headers, you need to use the optional context param when making each request. There's a good example here of sending a cookie. You shouldn't need to worry about the formatting too much; it's already sent in that format in the 'Set-Cookie' header, and I think getting everything before the semicolon is all you need to do. I would append that '\r\n' (line break) at the end though, since that might be necessary for the way PHP does things.

The 'Set-Cookie' HTTP response header is sent by the server; it contains a string and some metadata that the browser parses, and then the browser will send it back when it makes its next request to that site, putting the string in the 'Cookie' HTTP request header.
The Java code con.getHeaderFields().get("Set-Cookie"); is getting a list of all cookies set by the server. The code after that is parsing each one to get rid of everything after the first semicolon. And it appends all cookies being returned together with semicolons, since that's the format for sending multiple cookies in the next request. The con.setRequestProperty("Cookie", cookies); code is what actually sends the cookie in the next request.
I haven't used PHP much, so I'm not sure what the best way of doing things is, but from some basic searching, it seems like there's an $httpresponseheaders variable that gets populated when you do file_get_contents() (documentation). You should be able to check that array for any entries that start with 'Set-Cookie', and do the same parsing that the Java version does.
For sending back the headers, you need to use the optional context param when making each request. There's a good example here of sending a cookie. You shouldn't need to worry about the formatting too much; it's already sent in that format in the 'Set-Cookie' header, and I think getting everything before the semicolon is all you need to do. I would append that '\r\n' (line break) at the end though, since that might be necessary for the way PHP does things.

Thank you for all the help. Sorry I'm a bit uneducated as I'm a junior in high school working in a school project, but this thread has taught me tons and I appreciate it.
I'll continue with your advice and will notify if I'm encountering any problems.
Thanks all.

Haha, no problem. I discovered this repo back when I was in high school, and was also pretty lost, so I definitely understand feeling uneducated with all the stuff you have to know.
Good luck with your project!

Haha, no problem. I discovered this repo back when I was in high school, and was also pretty lost, so I definitely understand feeling uneducated with all the stuff you have to know.
Good luck with your project!

I received all of the 'Set-Cookie' values:
'_JSESSIONID=B6E5C38414091828BA465BC6854386EF; Path=/campus; Secure; HttpOnly
XSRF-TOKEN=a3929539-3be4-4ea1-a88d-6f93c4c732ef; Path=/campus
appName=; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/campus
appName=grossmont; Max-Age=10000000; Expires=Fri, 12-Jun-2020 21:55:02 GMT; Path=/
tool=; Path=/campus
selection=; Path=/campus
scheduleMode=; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/campus
walkinContext=; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/campus
walkinDate=; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/campus
filterEndYear=; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/campus
districtID=; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/campus
endYear=; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/campus
districtID=628; Path=/campus/portal
endYear=2020; Path=/campus/portal
calendarID=; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/
calendarID=-1; Path=/campus/portal
structureID=; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/campus/portal
calendarID=; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/campus
structureID=; Max-Age=0; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/campus
permCalendarID=-1; Path=/campus/portal
portalApp=student; Path=/campus
CampusCookies=!jwUueAlWlOAKophNWG2rg6K44hiZSZD/am3oRtvN4DpBbqmZtrPn5jwvCBxYfH0zoGNfGBpjbnkPPw==; expires=Tue, 18-Feb-2020 08:08:22 GMT; path=/; Secure_'
The Java parsing separates it by the semicolon, so I'm assuming I disregard the values after, such as 'Path =/campus' or 'Max-Age'.
Also, when sending the cookies in this format:
		$opts = array( 		  'http'=>array( 		    'method'=>"GET", 		    'header'=>"Accept-language: en\r\n" . 		              "Cookie: foo=bar\r\n" 		  ) 		);
Do I replace 'header' with 'Set-Cookie', or the key within?

When I implemented the code, I just took the "Set-Cookie" header value as-is and passed it directly into the "Cookie" line that you have in the header string. You shouldn't have to process the cookies on your end.
The server will probably disregard the repeating cookies, but if it works it works.

Do I replace 'header' with 'Set-Cookie', or the key within?

That $opts array is the headers you're sending to the server in a request, so the name of the 'Cookie' header should be left as-is. The 'Set-Cookie' header is only in responses, so you only need to worry about reading that header, never sending it. And 'header' is a standard name, along with 'http'. You don't need to change those when setting the $opts array.
The foo=bar part should be replaced with the data you got from the 'Set-Cookie' header in the login request. And when you make your initial login request, you don't need to include a 'Cookie' header (since at that point you probably won't have any cookie to send).

And do I keep the 'GET' method?

 The alternative would be a POST request, which has extra data in the 'body' of the request. The login request and the XML data request both put all the information they need right in the URL, so they don't need a body, so they are simple GET requests.

Also, there are many entries that are repeated, do I include all of them?

It looks like they have different paths, which means the different versions of the cookies should be sent based on the paths of different URLs the browser might request. Probably just include all of them, since that's what the original Java version is doing, and I assume that one still works.

When I implemented the code, I just took the "Set-Cookie" header value as-is and passed it directly into the "Cookie" line that you have in the header string. You shouldn't have to process the cookies on your end.
The server will probably disregard the repeating cookies, but if it works it works.

I guess that would work, as long as you join all the entries of 'Set-Cookie' together with semicolons. And if you said it worked for you, then it's probably fine.
Technically what's happening is that you're choosing not to parse the directives attached to each cookie, and instead simply sending them back to the server as if they were cookies. You are supposed to parse these out (normally it's the browser's job), but I guess it's safe to send these back as if they were extraneous cookies (they're semicolon-separated as if they were cookies, which might be by design(?)), so I don't think this will break anything.

The foo=bar part should be replaced with the data you got from the 'Set-Cookie' header in the login request. And when you make your initial login request, you don't need to include a 'Cookie' header (since at that point you probably won't have any cookie to send).

So, with all the retrieved data, the code structure would be :
`<?php
// Create a stream
$opts = array(
'http'=>array(
'method'=>"GET",
'header'=>"Accept-language: en\r\n" .
"
JSESSIONID=4A0DF6C36EBA6DFFF819110823C1C0F0;
XSRF-TOKEN=9eaf1070-b5d0-4985-aa5c-4048ef7f5606;
appName=;
appName=grossmont;
tool=;
selection=;
scheduleMode=;
walkinContext=;
walkinDate=;
filterEndYear=;
districtID=;
endYear=;
districtID=628;
endYear=2020;
calendarID=;
calendarID=-1;
structureID=;
calendarID=;
structureID=;
permCalendarID=-1;
portalApp=student;
"
)
);
$context = stream_context_create($opts);
// Open the file using the HTTP headers set above
$file = file_get_contents('myURL, false, $context);
?>`

I don't think line breaks are allowed within a single 'Cookie' header, since line breaks are used to separate each header from the next. And you can probably get rid of the empty cookies like "walkinContext".
Oh, and it's probably a bad idea to post the actual cookie data on a public forum like this, because that token is password-equivalent, meaning it can be used to log in to your account in see your grades, etc.

I don't think line breaks are allowed within a single 'Cookie' header, since line breaks are used to separate each header from the next. And you can probably get rid of the empty cookies like "walkinContext".
Oh, and it's probably a bad idea to post the actual cookie data on a public forum like this, because that token is password-equivalent, meaning it can be used to log in to your account in see your grades, etc.

Progress! We are so close!
After sending the cookies and retrieving the file_get_contents data, an error message is presented:
Campus Error District: Grossmont Union, CA - CA (grossmont) appServer c885ca-app003 Date: Feb 18, 2020 7:16:35 PM Error Token:<PHONE_NUMBER> An error has been reported on this page. This error may have been caused by invalid input. If so, please hit 'Back' and correct the mistake. If this is not the case, please record the error token given above and contact Campus Support Services. Include the steps necessary to reproduce the error and the recorded error token.
I am aware that my cookies are sending invalid inputs, I'm just not sure what that is.
My code structure for the cookies:
`		$opts = array(
'http'=>array(
'method'=>"GET",
'header'=>"Accept-language: en\r\n" ."Cookie: JSESSIONID=xyz; XSRF-TOKEN=xyz; appName=xyz; districtID=xyz; endYear=xyz; calendarID=xyz; permCalendarID=xyz; portalApp=xyz; CampusCookies=xyz==\r\n"
	  )
	);

             $context = stream_context_create($opts);

	 $userData = file_get_contents('myURL, false, $context);`

I also replaced the actual cookie data with 'xyz' placeholders.

When I implemented the code, I just took the "Set-Cookie" header value as-is and passed it directly into the "Cookie" line that you have in the header string. You shouldn't have to process the cookies on your end.
The server will probably disregard the repeating cookies, but if it works it works.

Did you mean something like this?:
`
$opts = array(
'http'=>array(
'method'=>"GET",
'header'=>"Accept-language: en\r\n" .
"Cookie: Set-Cookie: (string of cookies followed by semicolons)\r\n"
)
);
`

Okay I got the cookies to send!
The problem is,

I got the cookies to send! The problem now is that, instead of Infinite Campus displaying an error message or telling me to retrieve the correct url, the file_get_content of the url is just the login page of the campus, indicating that i haven't logged in yet.
I thought that the cookies would prevent that, so I think I am missing something.

You're still hardcoding the cookies, right? It's possible that one or more of the cookies have expired since you originally retrieved them, so you're being redirected to log in again. Maybe try getting a fresh set of cookies with a new login request. (Or automate the login-cookie-retrieval since you'll have to do that at some point anyway.)

You're still hardcoding the cookies, right? It's possible that one or more of the cookies have expired since you originally retrieved them, so you're being redirected to log in again. Maybe try getting a fresh set of cookies with a new login request. (Or automate the login-cookie-retrieval since you'll have to do that at some point anyway.)

I implemented a system where it retrieves a new fresh set of cookies each time the form is submitted.
Sent with GitHawk

`
$finalCookies";

    // Create a stream
		$opts = array(
		  'http'=>array(
		    'method'=>"GET",
		    'header'=>"Accept-language: en\r\n" ."Cookie: Set-Cookie: $finalCookies\r\n"

		  )
		);

     $context = stream_context_create($opts);
		 $userData = file_get_contents('https://grossmontca.infinitecampus.org/campus/prism?x=portal.PortalOutline&appName=grossmont', false, $context);
		 echo $userData;

	}else{
		echo 'unable to locate user';
		}

}

session_destroy();
?>

<input type="text" name="username" placeholder="Student ID"><br><br>
<input type="password" name="password" placeholder="InfiniteCampus Password"><br><br>
<input type="submit" name="submitIC">

`

             'header'=>"Accept-language: en\r\n" ."Cookie: Set-Cookie: $finalCookies\r\n"

You definitely shouldn't have the string "Set-Cookie" included in your 'Cookie' header. That's the name of the response header, and you only want the value from that, not the name.
The other guy mentioning he used the unmodified 'Set-Cookie' header meant he wasn't splitting each cookie by semicolon to remove the extra data like 'domain' and 'path', not that he was including the actual 'Set-Cookie' text.

<?php session_start();
//establish variables
$_SESSION['username'] = $_POST["username"];
$_SESSION['password'] = $_POST["password"];
//create cookies array
$cookies = array('');

//check if user is in infinite campus documentation
if(isset($_POST['submitIC'])){

	$verify = file_get_contents('https://grossmontca.infinitecampus.org/campus/verify.jsp?nonBrowser=true&username='.$_SESSION['username'].'&password='.$_SESSION['password'].'&appName=grossmont');
	if (strpos($verify, 'success') !== false) {

		$pageData = $http_response_header;

		foreach($pageData as $row){
			if(strpos($row, 'Set-Cookie:') !== false){

			if(strpos($row,'appName=;') || strpos($row,'tool=;') || strpos($row,'selection=;') || strpos($row,'scheduleMode=;') || strpos($row,'walkinContext=;') ||strpos($row,'walkinDate=;') ||strpos($row,'filterEndYear=;') ||strpos($row,'districtID=;') ||strpos($row,'endYear=;') ||strpos($row,'calendarID=;') ||strpos($row,'structureID=;')){

			}else{
				$cookie = explode('Set-Cookie:', $row);
				$cookie = explode(';', $cookie[1]);
				$cookie = $cookie[0];
				array_push($cookies, "$cookie;");
			}

			}
			else {
			}

		}

		$finalCookies = implode("",$cookies);
		echo "<br><br>$finalCookies<br><br>";

    // Create a stream
		$opts = array(
		  'http'=>array(
		    'method'=>"GET",
		    'header'=>"Accept-language: en\r\n" .
		    					"Cookie: $finalCookies\r\n"

		  )
		);

     $context = stream_context_create($opts);
		 $userData = file_get_contents('https://grossmontca.infinitecampus.org/campus/prism?x=portal.PortalOutline&appName=grossmont', false, $context);
		 echo $userData;

	}else{
		echo '<br>unable to locate user';
		}

}

session_destroy();
?>

<!-- form for infinite campus data-->
<form action="test.php" method="post">

	<input type="text" name="username" placeholder="Student ID"><br><br>
	<input type="password" name="password" placeholder="InfiniteCampus Password"><br><br>
	<input type="submit" name="submitIC">

</form>

To debug this, I would first try using the exact cookie the browser is using, and hardcode that.
I would try going into your browser, making sure you can successfully request the XML file, and then open the console (right click -> 'Inspect' -> 'Console' tab) and type document.cookie. The string it prints is the cookie that's being sent with the request, and you should set your $finalCookie to that string.
If that doesn't work, then there's something weird going on. Maybe try running the Java version and printing the cookie when it makes the XML request.
If it does work however, then you should compare the cookie from the browser that works to the cookie you're constructing that doesn't work. Obviously some stuff is different between every session like 'JSESSIONID', 'XSRF-TOKEN', etc. But put them side by side and see if there's any material differences in structure or content of other cookies that should be static.

Okay, awesome, it worked. There was definitely a difference in the cookies in the browser compared to mine. I copied and pasted the static variables, and I created two variables for CAMPUSCOOKIES and XSRF-TOKEN. There wasn't a JSESSIONID in the xml file cookies. This is what I used:

	foreach($pageData as $row){
			// if(strpos($row, 'Set-Cookie:') !== false){

			// if(strpos($row,'appName=;') || strpos($row,'tool=;') || strpos($row,'selection=;') || strpos($row,'scheduleMode=;') || strpos($row,'walkinContext=;') ||strpos($row,'walkinDate=;') ||strpos($row,'filterEndYear=;') ||strpos($row,'districtID=;') ||strpos($row,'endYear=;') ||strpos($row,'calendarID=;') ||strpos($row,'structureID=;')){

			// }elseif(strpos($row, 'CampusCookies=') == false){
			// 	$cookie = explode('Set-Cookie:', $row);
			// 	$cookie = explode(';', $cookie[1]);
			// 	$cookie = $cookie[0];
			// 	array_push($cookies, "$cookie;");
			// }elseif(strpos($row, 'CampusCookies=') !== false){
			// 	$cookie = explode('Set-Cookie:', $row);
			// 	$cookie = explode(';', $cookie[1]);
			// 	$cookie = $cookie[0];
			// 	array_push($cookies, "$cookie");
			// }

			// }
			// else {
			// }

			if(strpos($row, 'XSRF-TOKEN=')){
				$xsrf = explode('Set-Cookie:', $row);
				$xsrf = explode(';', $xsrf[1]);
				$xsrf = "$xsrf[0];";
				echo "<br>$xsrf<br>";
			}elseif(strpos($row, 'CampusCookies=')){
				$campusCookies = explode('Set-Cookie:', $row);
				$campusCookies = explode(';', $campusCookies[1]);
				$campusCookies = "$campusCookies[0]";
				echo "<br>$campusCookies<br>";			}

		}

		$finalCookies = implode("",$cookies);
		echo "<br><br>$finalCookies<br><br>";

    // Create a stream
		$opts = array(
		  'http'=>array(
		    'method'=>"GET",
		    'header'=>"Cookie: lang=en; lang=en; tool=; selection=; portalApp=student; $xsrf appName=grossmont; portalLang=en; $campusCookies\r\n"

		  )
		);

     $context = stream_context_create($opts);
		 $userData = file_get_contents('https://grossmontca.infinitecampus.org/campus/prism?x=portal.PortalOutline&appName=grossmont', false, $context);
		 echo $userData;

Good news and bad news. It didn't return an error or a redirect message. However, it return the login page again as the file_get_content data.

When you say it worked, do you mean the cookies you copied and pasted from the browser successfully retrieved the XML file? Or did that just get you the login page?

I took the cookies that the that displayed when I checked the loaded XML file on another tab, and I pasted the static ones in the array, then I created dynamic variables for the ones that changed.
'header'=>"Cookie: lang=en; lang=en; tool=; selection=; portalApp=student; $xsrf appName=grossmont; portalLang=en; $campusCookies\r\n"
After doing this, it didn't display an error message or redirect message, but it took me to the login page, which happens when you type the url in a tab without logging in prior to doing so.
I'm a bit uneducated but I am assuming that means that it is reading the cookies correctly and accepting them, but I am still missing some that give me access.

What I was suggesting was to copy the entire cookie string, and assign that to your $finalCookie variable. Without actually looking through the values and swapping out the dynamic ones (since the dynamic ones from the browser should still be valid when you make the request in PHP). Just to see if it works.
And after that, look for any discrepancies between your dynamically generated cookies and the browser's static cookie string. Maybe try getting the cookies via PHP, and then swapping those dynamic values into the browser-cookie-string, and making sure it still all works.
(By the way, the value of the XSRF-TOKEN shouldn't matter because XSRF tokens are sent via means other than request cookies. What XSRF tokens actually are and what they do is a thing to read about at a later date; you don't need to care about that right now.)

Thank you for all your help. I will attempt to run the program in Java and see the cookies that are returned.

Thank you for all your help. I will attempt to run the program in Java and see the cookies that are returned.

Did you get any success when running the program in java? I am not sure how to start with executing the code.


Game: download sdk at every initialize

TODO: download at every changes

Coverage decreased (-0.05%) to 39.826% when pulling 9eed27c6bc49ee6934676554618787ab2d9741cc on develop into 86e05cf3976ca760adbe63e757cfee8c9701fd74 on master.


Issue: question about agents

I want pass "chat_history" to the agents like :
...
agent = initialize_agent(tools, llm, agent=AgentType.OPENAI_FUNCTIONS, verbose=True)
response = agent.run({"input": {"chat_history": _message_to_tuple(history), "question": query}})

but got an error. how can i pass 'chat_history' to the agent?
Suggestion:
No response

could you pls tell what error u got?

could you pls tell what error u got?

how to pass chat_history to agent?

I set the ConversationalRetrievalChain with return_source_documents=True,

the tools contains a ConversationalRetrievalChain, and the chain set return_source_documents = True.

if i remove return_source_documents = True.. it will throw another err.


Email validation message added

🛠 Fixes Issue : ""

👨💻 Changes proposed:

✔ Check List (Check all the applicable boxes)

[x] My code follows the code style of this project.
[x] This PR does not contain plagiarized content.
[x] The title of my pull request is a short description of the requested changes.
[x] I am a participant of Hacktoberfest 2022.

📄 Note to reviewers

📷 Screenshots

Added email validation

Hey,
Please mention the issue number in the comment. Also, the name of the file should be either "signUp.html" or "sign_up.html". Please correct that.


Age Detection

Project-Guidance! 🎊🎈
Welcome! 🎉
Thank you for visiting our repository! We're excited to have you here. If you find this project useful, please consider giving it a star! ⭐
🎉 Have you reviewed the Contributing Guidelines? 🤔 (Required)

[X] YES
[ ] NO

📝 Description

Added a project that implements an age detection system using OpenCV's DNN module and a pre-trained Caffe model. The system detects faces in an image and predicts the age group of each detected face.

🔧 Fixes #issue_no (Required)
#1421
🏷 Initiative Participation (Required)
This pull request is associated with the following initiative. Please choose the one you are contributing to:

[ ] 🌸 GSSoC (Girl Script Summer of Code)
[X] 🌼 GSSoC 2024 Extd.
[X] 🍂 Hacktoberfest 2024
[ ] ❄ Script Winter of Code (SWoC)
[ ] ⛄ Winter of Blockchain (WoB)
[ ] 🙋♂ Personal
[ ] 📌 Other (please specify): _______________

@ramana2074 Mention Valid issue number.

#1405


Ability to set log level

Describe the solution you'd like
The ability to set the log level in the values file for the helm chart.
What is the added value?
Info logs are quite chatty and in my case, log level error would be enough.

You can already pass loglevel as an extraarg

Hi, thank you for pointing this out. Do you have a documentation reference on which arg I need to set?

@reakaleek you should take a look here
https://github.com/external-secrets/external-secrets/blob/main/main.go#L55
and here
https://github.com/external-secrets/external-secrets/blob/main/deploy/charts/external-secrets/values.yaml#L41
Just set logLevel: "error" and you should be fine

@reakaleek you should take a look here
https://github.com/external-secrets/external-secrets/blob/main/main.go#L55
and here
https://github.com/external-secrets/external-secrets/blob/main/deploy/charts/external-secrets/values.yaml#L41
Just set logLevel: "error" and you should be fine

Thank you very much!


Crawling working fine but downloading stucks

hi..
thanks for this wonderful script.
i am trying to download from goindex site, it crawls all files well but when i run next cell to download, it stucks and always remain at 0%. I doubled checked and saw some .txt files have been downloaded to destination and also folders have been created, but its not downlaoding actual big files, movies in my case.
can u fix that?

Could you leave the GD site for me, so I will try when I have time.


No module named "tenacity"

Anyone facing the same issue when executing the idea and project name? I installed Conda env all requirements are satisfied.
File "C:\ChatDev\run.py", line 19, in
from camel.typing import ModelType
File "C:\ChatDev\camel_init_.py", line 14, in
import camel.agents
File "C:\ChatDev\camel\agents_init_.py", line 15, in
from .chat_agent import ChatAgent
File "C:\ChatDev\camel\agents\chat_agent.py", line 17, in
from tenacity import retry
ModuleNotFoundError: No module named 'tenacity'

pip install tenacity

Sorry forgot to mention that all packages are installed 🙃

`
(ChatDev_conda_env) E:\git\ChatDev>pip install tenacity
Collecting tenacity
Obtaining dependency information for tenacity from https://files.pythonhosted.org/packages/f4/f1/990741d5bb2487d529d20a433210ffa136a367751e454214013b441c4575/tenacity-8.2.
3-py3-none-any.whl.metadata
Downloading tenacity-8.2.3-py3-none-any.whl.metadata (1.0 kB)
Downloading tenacity-8.2.3-py3-none-any.whl (24 kB)
Installing collected packages: tenacity
Successfully installed tenacity-8.2.3
(ChatDev_conda_env) E:\git\ChatDev>python3 run.py --task "[description_of_your_idea]" --name "[project_name]"
Traceback (most recent call last):
File "E:\git\ChatDev\run.py", line 19, in
from camel.typing import ModelType
File "E:\git\ChatDev\camel_init_.py", line 14, in
import camel.agents
File "E:\git\ChatDev\camel\agents_init_.py", line 15, in
from .chat_agent import ChatAgent
File "E:\git\ChatDev\camel\agents\chat_agent.py", line 17, in
from tenacity import retry
ModuleNotFoundError: No module named 'tenacity'
(ChatDev_conda_env) E:\git\ChatDev>
`

Oh I know what it is... On windows sometimes you can end up with a "python3.exe" somewhere else on your system. But when You make a venv (apparent even in conda) you get a python.exe. the same is not true in linux. so linux devs will forget to remove the 3 for the run command. so use
python run.py --task "[description_of_your_idea]" --name "[project_name]"
NOT
python3 run.py --task "[description_of_your_idea]" --name "[project_name]"
at least on windows

Oh I know what it is... On windows sometimes you can end up with a "python3.exe" somewhere else on your system. But when You make a venv (apparent even in conda) you get a python.exe. the same is not true in linux. so linux devs will forget to remove the 3 for the run command. so use
python run.py --task "[description_of_your_idea]" --name "[project_name]"
NOT
python3 run.py --task "[description_of_your_idea]" --name "[project_name]"
at least on windows

@TashaSkyUp
In fact you should always run every command with python3 instead python every time in your life. That is a good practice I think.

"python3.exe" does  not exist when you create the conda environment in windows.
(ChatDev_conda_env) E:\git\ChatDev\WareHouse\caption me_DefaultOrganization_20230910183821>where python3
C:\Users\Tasha\AppData\Local\Microsoft\WindowsApps\python3.exe

(ChatDev_conda_env) E:\git\ChatDev\WareHouse\caption me_DefaultOrganization_20230910183821>where python
C:\Users\Tasha\.conda\envs\ChatDev_conda_env\python.exe
C:\Python310\python.exe
C:\Users\Tasha\AppData\Local\Microsoft\WindowsApps\python.exe

So in fact for windows you get the error I mentioned and detailed above.

I'm getting the same error too

@mosobln were you able to resolve this?


WIid card and exlude value

Finally, a battery card!
Any change your card can get a wild card for entities and attributes?
And an exclude value option (some sensors like FIbaro can also be wired, the battery value is then defaulted to 4)
Cheers.
My cuurent setup:
card:
  show_header_toggle: false
  title: Sensor Battery Status
  type: entities
filter:
  exclude:
    - state: '4'
  include:
    - entity_id: sensor.fib_*_battery_level
      options:
        style: |
          :host {
          --paper-item-icon-color:
           {% if states(config.entity)|float <= 20 %}
             red
           {% elif states(config.entity)|float <= 35 %}
             orange
           {% else %}
             var( ---state-icon-color )
           {% endif %}
           ;
           }
sort:
  method: state
  numeric: true
  reverse: false
type: 'custom:auto-entities'

Interesing idea with the wildcard entity search! I'm slightly concerned it can be a bit slow if you have a lot of entities in your HA.
HA updates the cards "every time" (probably it is throttled) when any antity state changes (even if it's not used in the card). So in this case I would need to iterate over the entire collection of entities and get their state. But since you're saying that you have almost similar setup for the other card I will give it a try.
If it is ok perfwise I will add it in similar way as you have it in your example. If not I could do some kind of discovery mode in which list of found entities will be printed on the card - you would be able to copy it over to the configuration.
I don't see any problems with state filtering so I will add it for sure

FYI @Tom-ahawk this card works with auto-entities.  I'm using it right now like this:
- card:
    show_header_toggle: false
    title: Battery levels
    type: 'custom:battery-state-card'
    color_gradient:
      - '#ff0000'
      - '#ffff00'
      - '#00ff00'
  filter:
    exclude:
      - entity_id: '*_phone_*'
      - state: Ok
    include:
      - entity_id: sensor.*_battery_level
      - entity_id: sensor.*_battery_health
  show_empty: false
  sort:
    method: state
    numeric: true
  type: 'custom:auto-entities'

So @maxwroc its up to you if you want to implement this yourself in your card or simply point people to auto-entities as a solution to this problem.

It works fine with the 'custom:auto-entities' card.

Reopening this as there is more and more asks for the same

As an FYI the sort/collapse option also works wrapped in auto-entities:
- type: custom:auto-entities
     card:
       type: custom:battery-state-card
       title: "Battery Levels"
       secondary_info: last_updated
       sort_by_level: "asc"
       collapse: 4
     filter:
       include:
         - entity_id: sensor.*_battery_level
         - entity_id: sensor.*_battery

Yes all of the card-level options will be working

For others looking for similar solution: please check the filters functionality which was added in v1.3.0.
You don't need to use auto-entities any more!
Check the documentation https://github.com/maxwroc/battery-state-card#filter-groups
Example use:
https://github.com/maxwroc/battery-state-card#entity-filtering-and-bulk-renaming
- type: 'custom:battery-state-card'
  title: Filters
  sort_by_level: "asc"
  bulk_rename:
    - from: "Battery Level" # simple string replace (note: "to" is not required if you want to remove string)
      to: "sensor"
    - from: "/\\s(temperature|temp)\\s/" # regular expression
      to: " temp. "
  entities:
    # entities requiring additional properties can be added explicitly
    - entity: sensor.temp_outside_battery_numeric
      multiplier: 10
      name: "Outside temp. sensor"
  filter:
    include: # filters for auto-adding
      - name: entity_id # entities which id ends with "_battery_level"
        value: "*_battery_level"
      - name: attributes.device_class # and entities which device_class attribute equals "battery"
        value: battery
    exclude: # filters for removing
      - name: state # exclude entities above 99% of battery level
        value: 99
        operator: ">"


Adding support for Haskell

This PR adds REPL definitions for two Haskell REPLs: Stack's GHCi, and vanilla GHCi. More options could be added in the future - for example, adding a REPL that launches with Intero loaded, adding custom actions, etc. However, basic REPL inclusion is nice to have for now.
Please let me know if there are any other changes which should be made (i.e. tests, etc.). The changes in this PR simply (1) add the REPL definitions and (2) modify the docs to indicate that some Haskell support is available. I have tested the definitions manually, but not attempted any sort of automatic testing.

Awesome!
This is great, thanks for helping!
Regarding the automated tests, don't worry about it now... I'll surely add them later, but I still couldn't stop to make it happen..
Once again, thanks for the contribution!
Cheers!

Thanks! I'll probably try to add more features when I run into wanting them.

Also, major kudos for making it so dead-simple to add to this project. I went from wanting a REPL for Haskell to having a PR ready in something like 10 minutes.

Awesome! Please let me know if you have any questions or problems!
Cheers


Move mobile app menu to right

To the right of the default refresh button - far right of appBar
Most importantly to me this is consistent with the desktop which I think makes sense for this particular built in component.
But it would also allow an app logo glyph to appear first which I like and - when the back nav button is active - lets that button come and go without moving the anchor position for the main menu
Happy to be convinced otherwise of course.
Could also be an app bar prop like appMenuPosition that could take "none" to suppress I'd we really needed - but I would prefer it be consistent

https://github.com/exhi/hoist-react/pull/1217
https://github.com/exhi/toolbox/pull/216


create_feature_extractor  V.S. _utils.IntermediateLayerGetter

🐛 Describe the bug
When I used resnet50 in torchvision.models, I replaced BatchNorm2d with FrozenBatchNorm2d. Then I needed to extract the output of layer4, but I found that create_feature_extractor does not display FrozenBatchNorm2d. Then I used IntermediateLayerGetter and it displayed it properly.
As shown below:
import torch
from torchvision import models
from torchvision.models.feature_extraction import create_feature_extractor
from torchvision.models._utils import IntermediateLayerGetter

print(torch.__version__)  # 1.10.2+cu113

class FrozenBatchNorm2d(torch.nn.Module):
    """
    BatchNorm2d where the batch statistics and the affine parameters are fixed.

    Copy-paste from torchvision.misc.ops with added eps before rqsrt,
    without which any other models than torchvision.models.resnet[18,34,50,101]
    produce nans.
    """

    def __init__(self, n):
        super(FrozenBatchNorm2d, self).__init__()
        self.register_buffer("weight", torch.ones(n))
        self.register_buffer("bias", torch.zeros(n))
        self.register_buffer("running_mean", torch.zeros(n))
        self.register_buffer("running_var", torch.ones(n))

    def _load_from_state_dict(self, state_dict, prefix, local_metadata, strict,
                              missing_keys, unexpected_keys, error_msgs):
        num_batches_tracked_key = prefix + 'num_batches_tracked'
        if num_batches_tracked_key in state_dict:
            del state_dict[num_batches_tracked_key]

        super(FrozenBatchNorm2d, self)._load_from_state_dict(
            state_dict, prefix, local_metadata, strict,
            missing_keys, unexpected_keys, error_msgs)

    def forward(self, x):
        # move reshapes to the beginning
        # to make it fuser-friendly
        w = self.weight.reshape(1, -1, 1, 1)
        b = self.bias.reshape(1, -1, 1, 1)
        rv = self.running_var.reshape(1, -1, 1, 1)
        rm = self.running_mean.reshape(1, -1, 1, 1)
        eps = 1e-5
        scale = w * (rv + eps).rsqrt()
        bias = b - rm * scale
        return x * scale + bias

model = models.resnet50(pretrained=True, norm_layer=FrozenBatchNorm2d)
model1 = create_feature_extractor(model, return_nodes={'layer4': '0'})
model2 = IntermediateLayerGetter(model, return_layers={'layer4': '0'})

# model1.load_state_dict(model2.state_dict())  # error !

print(model1)
print(model2)

The partial output of create_feature_extractor is as follows:
ResNet(
  (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (relu): ReLU(inplace=True)
  (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
  (layer1): Module(
    (0): Module(
      (conv1): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (relu): ReLU(inplace=True)
      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (downsample): Module(
        (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      )
    )

The partial output of IntermediateLayerGetter is as follows
(layer4): Sequential(
    (0): Bottleneck(
      (conv1): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn1): FrozenBatchNorm2d()
      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
      (bn2): FrozenBatchNorm2d()
      (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn3): FrozenBatchNorm2d()
      (relu): ReLU(inplace=True)
      (downsample): Sequential(
        (0): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): FrozenBatchNorm2d()
      )
    )

So, why does create_feature_extractor not work as expected. And, I wonder if there is a built-in function with the same functionality as FrozenBatchNorm2d above.
Versions
PyTorch version: 1.10.2+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A
OS: Microsoft Windows 10 专业版
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A
Python version: 3.8.5 (default, Sep  3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.19041-SP0
Is CUDA available: True
CUDA runtime version: 11.1.74
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.22.1
[pip3] numpydoc==1.1.0
[pip3] torch==1.10.2+cu113
[pip3] torch-tb-profiler==0.2.0
[pip3] torchaudio==0.10.2+cu113
[pip3] torchfile==0.1.0
[pip3] torchsummary==1.5.1
[pip3] torchvision==0.11.3+cu113
[pip3] torchviz==0.0.2
[conda] blas                      1.0                         mkl
[conda] cudatoolkit               11.1.1               heb2d755_9    conda-forge
[conda] mkl                       2020.2                      256
[conda] mkl-service               2.3.0            py38hb782905_0
[conda] mkl_fft                   1.2.0            py38h45dec08_0
[conda] mkl_random                1.1.1            py38h47e9c7a_0
[conda] mypy-extensions           0.4.3                    pypi_0    pypi
[conda] numpy                     1.22.1                   pypi_0    pypi
[conda] numpydoc                  1.1.0              pyhd3eb1b0_1
[conda] torch                     1.10.2+cu113             pypi_0    pypi
[conda] torch-tb-profiler         0.2.0                    pypi_0    pypi
[conda] torchaudio                0.10.2+cu113             pypi_0    pypi
[conda] torchfile                 0.1.0                    pypi_0    pypi
[conda] torchsummary              1.5.1                    pypi_0    pypi
[conda] torchvision               0.11.3+cu113             pypi_0    pypi
[conda] torchviz                  0.0.2                    pypi_0    pypi

@Alokia Thanks for reporting.
This is because FX removes all class information from modules. This is also the reason why we have not migrated yet the existing models to use the util. See https://github.com/pytorch/vision/issues/4540#issuecomment-994920254 for more info.

Thank you for your reply, I will look into it

@Alokia Thanks for reporting.
This is because FX removes all class information from modules. This is also the reason why we have not migrated yet the existing models to use the util. See #4540 (comment) for more info.

@Alokia Thanks for reporting.
This is because FX removes all class information from modules. This is also the reason why we have not migrated yet the existing models to use the util. See #4540 (comment) for more info.

Thank you for your reply, I will look into it

@Alokia Thanks for reporting.
This is because FX removes all class information from modules. This is also the reason why we have not migrated yet the existing models to use the util. See #4540 (comment) for more info.

Thank you for your reply, I will look into it

@Alokia I think one of the reasons the FX trace is giving you grief is because the specific definition of FrozenBatchNorm2d is custom (not TorchVision's) and thus is not automatically considered a leaf node by:
https://github.com/pytorch/vision/blob/a7a3675678cce66751050619d0c86c2ceafd7295/torchvision/models/feature_extraction.py#L229-L236
Try doing the following and it should be maintained:
tracer_kwargs = {
	"leaf_modules": [FrozenBatchNorm2d],
}
model = models.resnet50(pretrained=True, norm_layer=FrozenBatchNorm2d)
model1 = create_feature_extractor(model, return_nodes={'layer4': '0'}, tracer_kwargs=tracer_kwargs)

@jdsgomes Do you think we should update the code to always include math and TorchVision ops even when someone provides tracer_kwargs?
@alexander-soare just wanted to confirm with you that the behaviour highlighted here by @Alokia is expected. Let me know if you have any thoughts. Thanks!

@datumbox I think we touched on this topic when this change was introduced and unfortunately I cannot think of an elegant solution.
From one side  I don't like the idea of adding stuff to the parameters provided (adding ops and math), this might be misleading to the user. But on the other side I agree that this is the expected behaviour in most cases and would simplify things for most users, so I suggest an alternative option:
add an extra argument enforce_default_autowrap with default value True which will add the math and torchvision.ops to the tracer_kwargs

@jdsgomes I remember...  :(
Do you think there is a valid reason why someone would want to skip the default autowraps? Does it make sense to make them "mandatory"? From memory PyTorch core puts all the build-in nn.Module in the list and I believe that's hardcoded (though we need to check). If that's true, then we might have a valid reason to do the same and always append our internal ops in the list. Thoughts?

@datumbox  That is a good point, I can see that is indeed the case  in Pytorch core, so we can align our approach and make it mandatory by always.
I don't see a good reason for skipping the default auto-wraps.

@jdsgomes is it on your radar to add it or shall we create an issue?

@datumbox @jdsgomes for PyTorch core, the idea is that the user should make their own custom tracer with an override to is_leaf_module (citing this section in the documentation). So, although it is "mandatory", there is a recommended way to override it for the hardcore users. I think it would be really ugly to do something similar here but my (nit) point is that it's not necessarily correct to say that "if we make it mandatory we are aligned with core". Overall I think we should either just make it mandatory and wait till someone complains, OR:

Leave it the way it is (not mandatory)
Add a caution in the docstring explanation for that kwarg. People will see it if they dug deep enough to find that kwarg in the first place.
Expose some variant of _get_leaf_modules_for_ops to the user so they have a handy way of recovering them.

It is indeed in my radar @datumbox.
@alexander-soare thanks for the clarification. I am still leaning towards making it mandatory, and if we later decide otherwise   (people come up with use cases for that), we can change the behaviour in a non BC-breaking way by introducing a default flag for instance.


Update compiler version and dependency versions in Aztec contract example

Examples uses 0.18.0
Latest Noir release at 0.33.0 at the time of writing.

Thanks for the poke!
This is actually pending for my question in the Noir Discord: https://discord.com/channels/1113924620781883405/1275797432365482044/1280761732478013441
I kind of gave up on it, as nobody responded, and decided to try later somewhere in the future.

After some discussions and support, I finally got both the stand-alone and contract usage working. Thanks to all Aztec people for the help.
The Noir examples have been updated ot use version 0.35.0.


node-sass compiles with full uri in package version

After upgrading to Node 8, node-sass package.json is generated to include the full uri to the downloadable .tar file in its "version" parameter. (We are using an npm proxy internally)
This causes the version to be semantically incorrect, which causes the further installation to fail finding the correct binary for lib-sass online, which then triggers a complete build which in turn also is corrupted.

NPM version (npm -v): 5.0.3
Node version (node -v): 8.1.1 (also tried 8.0.0)
Node Process (node -p process.versions):
http_parser: '2.7.0',
node: '8.1.1',
v8: '5.8.283.41',
uv: '1.12.0',
zlib: '1.2.11',
ares: '1.10.1-DEV',
modules: '57',
openssl: '1.0.2l',
icu: '59.1',
unicode: '9.0',
cldr: '31.0.1',
tz: '2017b'
Node Platform (node -p process.platform): win32
Node architecture (node -p process.arch): x64
node-sass version (node -p "require('node-sass').info"): node-sass\thttp://npmproxy:8089/repository/npm/node-sass/-/node-sass-4.5.3.tgz\t(Wrapper)\t[JavaScript]\r\nlibsass  \t3.5.0.beta.2\t(Sass Comp
iler)\t[C/C++]
npm node-sass versions (npm ls node-sass):  N/A

I'm sorry but I don't understand the issue. Please provide the install out
that shows the issue.
Node sass hasn't changed anything for node 8 so the issue is either
environmental on your end or an issue with npm 5/node 8.
On 14 Jun. 2017 5:28 pm, "Jørn Are Hatlelid"<EMAIL_ADDRESS>wrote:

After upgrading to Node 8, node-sass package.json is generated to include
the full uri to the downloadable .tar file in its "version" parameter. (We
are using an npm proxy internally)
This causes the version to be semantically incorrect, which causes the
further installation to fail finding the correct binary for lib-sass
online, which then triggers a complete build which in turn also is
corrupted.

NPM version (npm -v): 5.0.3
Node version (node -v): 8.1.1 (also tried 8.0.0)
Node Process (node -p process.versions):
http_parser: '2.7.0',
node: '8.1.1',
v8: '5.8.283.41',
uv: '1.12.0',
zlib: '1.2.11',
ares: '1.10.1-DEV',
modules: '57',
openssl: '1.0.2l',
icu: '59.1',
unicode: '9.0',
cldr: '31.0.1',
tz: '2017b'
Node Platform (node -p process.platform): win32
Node architecture (node -p process.arch): x64
node-sass version (node -p "require('node-sass').info"):
node-sass\thttp://npmproxy:8089/repository/npm/node-sass/
-/node-sass-4.5.3.tgz\t(Wrapper)\t[JavaScript]\r\nlibsass
\t3.5.0.beta.2\t(Sass Comp
iler)\t[C/C++]
npm node-sass versions (npm ls node-sass): N/A

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/sass/node-sass/issues/2018, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAjZWGwtfcL3XSiAB1G3TcBF3t82qaA3ks5sD4u6gaJpZM4N5fPZ
.

the node-sass package.json
{
"_args": [
[
"node-sass@http://npmproxy:8089/repository/npm/node-sass/-/node-sass-4.5.3.tgz",
"D:\myproject"
]
],
"_from": "node-sass@http://npmproxy:8089/repository/npm/node-sass/-/node-sass-4.5.3.tgz",
"_id"<EMAIL_ADDRESS>"_inBundle": false,
"_integrity": "sha1-0JydEXlkEjnRuX/8YjH9zsU+FWg=",
"_location": "/node-sass",
"_phantomChildren": {},
"_requested": {
"type": "remote",
"raw": "node-sass@http://npmproxy:8089/repository/npm/node-sass/-/node-sass-4.5.3.tgz",
"name": "node-sass",
"escapedName": "node-sass",
"rawSpec": "http://npmproxy:8089/repository/npm/node-sass/-/node-sass-4.5.3.tgz",
"saveSpec": "http://npmproxy:8089/repository/npm/node-sass/-/node-sass-4.5.3.tgz",
"fetchSpec": "http://npmproxy:8089/repository/npm/node-sass/-/node-sass-4.5.3.tgz"
},
"_requiredBy": [
"#DEV:/"
],
"_spec": "http://npmproxy:8089/repository/npm/node-sass/-/node-sass-4.5.3.tgz",
"_where": "D:\myproject",
"author": {
"name": "Andrew Nesbitt",
"email"<EMAIL_ADDRESS>"url": "http://andrew.github.com"
},
"bin": {
"node-sass": "bin/node-sass"
},
"bugs": {
"url": "https://github.com/sass/node-sass/issues"
},
"dependencies": {
"async-foreach": "^0.1.3",
"chalk": "^1.1.1",
"cross-spawn": "^3.0.0",
"gaze": "^1.0.0",
"get-stdin": "^4.0.1",
"glob": "^7.0.3",
"in-publish": "^2.0.0",
"lodash.assign": "^4.2.0",
"lodash.clonedeep": "^4.3.2",
"lodash.mergewith": "^4.6.0",
"meow": "^3.7.0",
"mkdirp": "^0.5.1",
"nan": "^2.3.2",
"node-gyp": "^3.3.1",
"npmlog": "^4.0.0",
"request": "^2.79.0",
"sass-graph": "^2.1.1",
"stdout-stream": "^1.4.0"
},
"description": "Wrapper around libsass",
"devDependencies": {
"coveralls": "^2.11.8",
"eslint": "^3.4.0",
"istanbul": "^0.4.2",
"mocha": "^3.1.2",
"mocha-lcov-reporter": "^1.2.0",
"object-merge": "^2.5.1",
"read-yaml": "^1.0.0",
"rimraf": "^2.5.2",
"sass-spec": "3.5.0-1"
},
"engines": {
"node": ">=0.10.0"
},
"files": [
"bin",
"binding.gyp",
"lib",
"scripts",
"src",
"test",
"vendor"
],
"gypfile": true,
"homepage": "https://github.com/sass/node-sass",
"keywords": [
"css",
"libsass",
"preprocessor",
"sass",
"scss",
"style"
],
"libsass": "3.5.0.beta.2",
"license": "MIT",
"main": "lib/index.js",
"name": "node-sass",
"nodeSassConfig": {
"binarySite": "https://github.com/sass/node-sass/releases/download"
},
"repository": {
"type": "git",
"url": "git+https://github.com/sass/node-sass.git"
},
"scripts": {
"build": "node scripts/build.js --force",
"coverage": "node scripts/coverage.js",
"install": "node scripts/install.js",
"lint": "eslint bin/node-sass lib scripts test",
"postinstall": "node scripts/build.js",
"prepublish": "not-in-install && node scripts/prepublish.js || in-install",
"test": "mocha test/{*,/}.js"
},
"version": "http://npmproxy:8089/repository/npm/node-sass/-/node-sass-4.5.3.tgz"
}
The install output
node scripts/install.js
Downloading binary from https://github.com/sass/node-sass/releases/download/vhttp://npmproxy:8089/repository/npm/node-sass/
-/node-sass-4.5.3.tgz/win32-x64-48_binding.node
Cannot download "https://github.com/sass/node-sass/releases/download/vhttp://npmproxy:8089/repository/npm/node-sass/-/node-
sass-4.5.3.tgz/win32-x64-48_binding.node":
HTTP error 404 Not Found
Hint: If github.com is not accessible in your location
try setting a proxy via HTTP_PROXY, e.g.
export HTTP_PROXY=http://example.com:1234 or configure npm proxy via
npm config set proxy http://example.com:8080

I just did a downgrade to node 6.1.1 (npm 5.0.3) and got the same result.

Did you try https://github.com/sass/node-sass/issues/1991#issuecomment-306437979

@nschonni Thank you, that looks like the same issue. I flushed all cache, deleted node_modules and package-lock.json, installed lastest node & npm and it ran like a charm. :)


Implement locker support for redis

Currently Consul is only supported for perform sequential updates. would be nice if we could accomplish this with redis as well.

Gonna close this for now as it is not really needed by anyone


v7.4 updates

Added Raven and Beast of Sacrifice bosses
Added Thelema
Sorted the rest of the valkyries prior to 4.4

@PaeP3nguin poke :(

Sorry, been busy! Thanks for the PR


Ideas for CHAOS

I have some ideas for CHAOS that you can iplement in the future if you want .
1st - Create switches on chaos
This means if user wants to create a backdoor without having to stop on every option all he/she have to do is to invoke the switches .
ex : ./chaos --g --lhost <IP_ADDRESS> --lport 7777 --exe testfile.exe
-g=generate
To generate these switches all you have to do  is put some if statements at the start of your script .
$0 = --g
$1= --lhost
$2= ip
etc ....
In bash is easy to do , but in go i have no idea .
2nd - Create interface with metasploit framework listeners (i do not think if it is possible) , but it would be a nice idea if implemented .

Hi, thanks for the ideas. Maybe in the future i will implement something.


Add mbed describe functionality

When developing firmware that can be programmed using DFU it is very helpful to know the exact version and state of the firmware that is programmed on a device / compiled in a DFU package.
For this, I typically use the git describe --tags --match "v*" --dirty command, which will just output the version tag if the last commit was tagged with a version (useful for "production" releases), but also the amount of commits since the last tag, the last commit hash and whether files in the repository where changed since the last commit (super useful during development).
However, since mbed libraries are ignored by git, changes to files in mbed libraries do not get reflected by git describe. mbed status can show changed files in libraries, but it does not give any information if a different revision of a library is checked out than what is defined in the .lib file. Also, for some reason mbed status does not show changes to files in the main repository.
An mbed describe command with similar options to git describe would be a very helpful feature to solve this problem.
Alternatively, having mbed status show "out of sync" libraries or having a "dry-run" option to mbed deploy would allow to build up a complete state description using the existing commands.
Having the mbed describe command would have the big benefit of supporting Mercurial repositories and it would make it possible to easily ignore files outside of the scope of the firmware but still checked into the repository (we typically put the mbed-based firmware and PCB design files in the same repo, but there is no point in flagging the firmware as dirty if a change was made to e.g the PCB layout).

Issue header has missing or incorrectly filled in fields! Please fix.


spatialBeta / layerControllerBeta views

Towards #830
Closes #1399

Background
I went back and forth about how to merge the "prototype" implementation of the updated spatial and layerController views from #1399 . I had initially decided against redundant copied code like this, but i now think it is the most practical way forward.
The current spatial and layerController views are very complex, and there are a lot of edge cases that will need to be re-implemented or at least manually tested for the HuBMAP data portal, so I would prefer the flexibility of the parallel "beta" implementations until those can reach feature parity with our current implementations, at which point we can use a view config upgrade function to migrate completely to the beta versions (and remove the Beta suffixes).
TODO

[x] Copy bitmaskLayer and shaders from #1399
[x] Tooltips for multi-obsType segmentations (using https://github.com/vitessce/vitessce/pull/1458)
[x] Convert photometricInterpretation to a per-imageLayer coordination type
[x] Re-implement channel controller
[x] MultiLegend
[x] Ensure reset viewport button works
[x] Support CoordinationScope instances here; add test https://github.com/vitessce/vitessce/blob/4b8260330ac8e44eddad6821d8717cd5c5046340/packages/config/src/VitessceConfig.js#L766
[x] Updated IMS demo
[x] ensure layer level properties fall back to view level properties
[x] Usage of coarser (_Channel -> _Layer -> view) coordination values when finer ones are null
[x] Demo with OME-Zarr (image, obsSegmentations)
[x] Demo with two image layers (OME-TIFF/OME-Zarr/both) and physicalSizeScaling / support for modelMatrix
[x] Simplify image data loader return values (add extra value like instance in LoaderResult.data.image.{loaders,meta,+instance} that the _Beta views can use while existing views ignore - eventually can make this value the root image value once out of beta)
[x] Example of global Z/T sliders
[x] Re-implement 3D controls
[ ] Example of same volume side by side with different 3D rendering mode

TODO: which of the following should be implemented in follow-up PRs?

[ ] Figure out solution for auto-initialization
[ ] ~Update segmentation channels UI to be more like an image channel UI?~
[ ] Re-implement molecules support (spatial and controller)
[ ] Automatic upgrade from spatial to spatialBeta
[ ] Automatic upgrade from layerController to layerControllerBeta
[ ] Cell set-based obs color encoding
[ ] Support pseudo-segmentations using obsLocations data for the specified obsType. Support one pseudoSegmentationLayer? (Can't think of a case when multiple would make sense)
[ ] Support a moleculeLayer using obsLocations and obsLabels data for the specified obsType. Support one layer of molecules? (Can't think of a case when multiple would make sense)

Related to #1576 and #1561 and #1458

Change List

Checklist

[ ] Ensure PR works with all demos on the dev.vitessce.io homepage
[ ] Open (draft) PR's into vitessce-python and vitessce-r if this is a release PR
[ ] Documentation added or updated

Add obsPoints obsSpots and obsCentroids data types to enable auto initialization. Deprecate obsLocations?


Do we need addition to 2701?

/* 2701 */
user_pref("network.cookie.cookieBehavior", 1);

is ignored, unless
user_pref("browser.contentblocking.category", "custom");

browser.contentblocking.category is missing in user.js  and thus set by default to standard.
ATM user must manually change in GUI to custom.
Cheers

if you change your cookie behavior to 1, the custom setting is automatically applied. Try it on a new profile.

No, it is not.
I have tried with a new profile before posting an issue 😉 .
Maybe this is new to latest FF.

Lemme do some tests: I have ESR60, ESR68, FF60-FF73 portables with practically vanilla profiles all here as my test suite

Its not only the latest FF, it looks like FF 70 also.

I cannot replicate
🐧 info for reference: network.cookie.cookieBehavior

0=Accept cookies and site data
1=(Block) All third-party cookies
2=(Block) All cookies
3=(Block) Cookies from unvisited websites
4=(Block) Cross-site and social media trackers (FF63+) (default FF69+)

🐧 Tests / STR
FF71-73 (default 4)

tab1: about:preferences#privacy = standard
tab2: about:config change pref to 1
tab1: automagically changes to custom - change it back to standard
tab2: about:config change pref to 2
tab1: automagically changes to custom
not going to test 0 or 3
Interestingly, if you reset the pref (in about:config), it stays in custom mode (even after a restart) even though nothing is different. This might be by design and is immaterial IMO: it's just cosmetic.

**ESR68 (default = 0)

results with 1 and 2 and 4
they all automagically change to custom

Prefs in about:config are (often) tied to a prefs-observer which then does things like "automatically" change this thing to "custom" etc. But that's not (or not always) the case when a pref is applied from a user.js.
So I guess the STR would be to let the user.js apply the cookie pref and then check if the GUI correctly changed to custom

I'll let someone else test it: I need sleep. But it's still cosmetic and we don't want to add browser.contentblocking.category to the user.js

Here is the video: https://www.dropbox.com/s/at740nllocidetp/ffvideo.mp4?dl=0
Windows 10 up-to-date, FF 71 x64.

You can also simply test with your working profile (where u have user.js).
Change to Standard... restart FF. Is should be Custom, due to user.js, but it persist to Standard.

BTW: your video didn't show the value of the pref you were adding, but we'll assume it was not 4 :)
It's just cosmetic. You can fix it manually - just click custom and restart the browser. Adding browser.contentblocking.category is a bit messy. The UI might confuse people - I'll have to re-check exactly what is shown to determine what to do here.

good lord I need some sleep
I think (99% sure) that we can just enforce browser.contentblocking.category = custom

when in custom display, the info displayed is always correct (i.e what is being blocked = exactly what the prefs are) - it doesn't matter if the configuration matches the default standard setup

see my point about "Interestingly, if you reset the pref (in about:config), it stays in custom mode (even after a restart) even though nothing is different."

when not in custom display and we end up with crssi's situation, the info displayed is incorrect (i.,e it says it is only blocking "cross site and tracking cookies" when in really it could be blocking all cookies)

So the only way to ensure the blocking info is correct is to enforce custom
@earthlng Does this sound right to you?

PPS: Thanks crssi 💋

BTW: your video didn't show the value of the pref you were adding, but we'll assume it was not 4 :)

Not needed, since it shows that I have started a clean vanilla profile, and then I have used user.js, which enforces 1. 😉

i.,e it says it is only blocking "cross site and tracking cookies" when in really it could be blocking all cookies

I have think off that it might be only GUI fckdup, but in reality it is not blocking as long as it is Standard... I have tested it over https://www.whatismybrowser.com/detect/are-third-party-cookies-enabled

I think (99% sure) that we can just enforce browser.contentblocking.category = custom

I will enforce it in my overrides if its not enforced by user.js.

PPS: Thanks crssi 💋

least I can do ❤

FFX: 73; new profile.
browser.contentblocking.category is a hidden pref. If it is unset (not in user.js/prefs.js) and user_pref("network.cookie.cookieBehavior", 1); is in the user.js browser.contentblocking.category becomes custom. If it is set, it is not touched.

At first I wasn't able to reproduce it ie I put a user.js with the cookie pref = 1 into a new profile, started FF and the category was correctly set to custom.
BUT once you change it to "standard", it sticks!
What I think is happening is that initially on FF start the user.js is correctly applied but then at some point FF sees .category=standard and resets all the relevant contentblocking prefs including the cookiebehavior.
I tested FF71 and todays Nightly and it's the same behavior in both. Ideally we should let mozilla know and get this fixed upstream but until then, yes I think we should enforce the category pref to custom.
Thanks @crssi, good find!
@Thorin-Oakenpants would you mind opening a ticket? thank you ;)

open a ticket where? at moz?

At first I wasn't able to reproduce it ie I put a user.js with the cookie pref = 1 into a new profile, started FF and the category was correctly set to custom. BUT once you change it to "standard", it sticks!

  The cookie behaviour pref wasn't being applied, which was alarming - and I aimed to come back to it. Now it makes sense when you said and resets all the relevant contentblocking prefs including the cookiebehavior.

Looks like your approach to new profile might be different to mine.
STR:

Close FF
Delete FF profile folders, that is %appdata%\Mozilla\Firefox
Start FF, so profile folder with initial files are created.
Close FF
Add user.js into profile folder
Start FF... there is Standard instead of Custom, despite the value 1 in the user.js

Here it is reproducible every single time.
Cheers

Here it is reproducible every single time.

Yeah, but that doesn't reproduce the problem that earthlng wants me to lodge with mozilla.
I tested it twice (on the same profile). My test suite of portables are vanilla profiles with a user.js already there with some tweaks. So I simply used the existing user.js to add one line - which actually confirmed it. So the test wasn't flawed in the first test (luckily because ETP had never been changed: as earthlng found out: he had trouble reproducing), And the subsequent test I did on the ETP modified profile, showed it getting stuck in standard including the actual cookie setting.
Due to the two different results, like earthlng, I would have dug deeper.

@earthlng How would you frame this issue for mozilla: since you're the one with the exact STR? Is this also an enterprise policy option?

but then at some point FF sees .category=standard and resets...

I guess resetting the pref also fixes this: you could use that in the STR to show it was the problem.
In cfg ... pref(), user_pref(), sticky_pref() and lockPref() .. I though there was a clear_pref or something?

FYI: OT: reddit link

open a ticket where? at moz?

yes. Pretty please ... :)

Looks like your approach to new profile might be different to mine.

correct, I didn't do 3 + 4 in your STR. In step 3 the category gets set to "standard" and apparently once that's set, it sticks.
If you do 5 before 3, it'll correctly set the category to custom.
It's easier to test with FF portable because you can just create an empty "profile" folder under "Data" and put the user.js in there before you start FF for the 1st time with that profile.
If you want to test it your way, you can do 1, 2 + 3 to let FF create the randomly named profile folder, then delete everything in there, put the user.js in and then start FF again.

you can just create an empty "profile" folder under "Data" and put the user.js in there before you start FF for the 1st time with that profile

But I reproduced without an empty profile. In fact my profile (portable 71) is was week old and had been opened dozens of times, including going to the options UI. Hope you weren't saying an empty profile was required in STR?

You don't need and empty profile.
Just change to Standard and restart FF. It will stay in standard and will not block 3rd party cookies, despite the value 1 in user.js

The main issue is that once browser.contentblocking.category is set to standard (either manually or otherwise), all those ETP prefs are no longer honored when set from a user.js. ("no longer honored" = I'm pretty sure they get initially applied during startup but then later reset when FF sees .category="standard")
And it's not the UI page itself that does the resetting, it happens during startup

AFAICT, in FF71 the affected ETP prefs are all of these:

network.cookie.cookieBehavior
privacy.trackingprotection.pbmode.enabled
privacy.trackingprotection.enabled
privacy.trackingprotection.socialtracking.enabled
privacy.trackingprotection.fingerprinting.enabled
privacy.trackingprotection.cryptomining.enabled

Since network.cookie.cookieBehavior is the only pref out of those that we still have in the user.js, IMO we can just add the category pref to 2701 and keep it there until after the next ESR gets released, assuming they'll fix the issue by then.

FYI ESR 68.3.0 has the same problem.

The main issue is that once browser.contentblocking.category is set to standard (either manually or otherwise), all those ETP prefs are no longer honored when set from a user.js. ("no longer honored" = I'm pretty sure they get initially applied during startup but then later reset or overwritten when FF sees .category="standard")

^^ This.

And it's not the UI page itself that does the resetting, it happens during startup

Yes. In my test about 24 hrs ago, the cookie behavior pref wasn't applied. I'd start FF, go straight to about:config and check

The main issue is that once browser.contentblocking.category is set to standard (either manually or otherwise)

I don't think this is quite true anymore. A brand new profile sets this to standard. Anyway, I'm pinging the moz guys

I don't think this is quite true anymore.

why not?
The main problem is when someone applies the user.js to a profile with ETP=standard, the cookie pref won't get set

nvm

FYI: hidden some comments to make for easier reading by moz engineers

FYI: I've passed this info onto someone at Mozilla - up to them to forward it to someone to consider and deal with it

FYI: 1607249 - Enterprise is not affected (which was what I was worried about), but outside of runtime is - e.g. user.js.
This is a WONTFIX which I don't fully understand (is user.js not supported? is it unofficial? not AFAIK). At the time, they made sure enterprise was OK but didn't think of all the methods, which I find really frustrating.
 The only conclusion I can draw from this, is that user.js (and cfg etc) are second class citizens and no longer supported or even considered (especially now Enterprise Policies exist)


Deprecated GET /branch/{branchid}

[x] I've added a new note in the CHANGELOG.md file, according to docs:
https://iver-wharf.github.io/#/development/changelogs/writing-changelogs

Summary

Moved handler for GET /branch/{branchid} into new package internal/deprecated
Added @deprecated flag to swaggo comment

Motivation
Closes #74
There were other endpoints planned for deprecation, such as POST /project/search, but I'm leaving that for the PR of #71.
The other "Not implemented" endpoints in the build.go file are planned for adding implementation to in https://github.com/iver-wharf/wharf-api/milestone/1

Why are we moving this?
For a deprecated method/route/api i would expect removal or an ignore.

It was moved to internal/deprecated package to get it out of the way. First we mark it as deprecated, and then in a later version bump we remove it.


Add CelerData Integration documentation to navigation

Is your feature request related to a problem? Please describe.
Currently the CelerData Integration documentation, located at docs/integrations/unity-catalog-celerdata.md, is missing from the navigation and with that from the documentation.
Describe the solution you would like
Add it to the mkdocs.yml configuration.
Additional context
See comment in #572 for details.

take


adding dockerfile

simple dockerfile which builds the project and serves the static webpage via nginx
Either this or #422
The other one hosts an api and doesn't use nginx but adds a new dependency
https://docs.docker.com/get-docker/
The following checks if docker image is running and stops it, then builds the image and lastly starts it on port 1000
name=damage_calc
img=unsupo/damage-calc
docker ps -a | grep ${name} > /dev/null 2>&1 && { docker stop ${name} > /dev/null 2>&1; docker rm ${name} > /dev/null 2>&1; }
docker build -t ${img} .
docker run -p 1000:3000 --name ${name} -d ${img}

@KrisXV i've added instructions to get this to work.

@KrisXV i've added instructions to get this to work.

docker run -p 1000:3000 --name ${name} -d ${img}

nginx listens on 80 by default, not 3000

You are right, i did neglect to port forward the right port in my example script here, but if you pull the prs you'll see they do in fact work.

docker run -p 1000:3000 --name ${name} -d ${img}

nginx listens on 80 by default, not 3000

You are right, i did neglect to port forward the right port in my example script here, but if you pull the prs you'll see they do in fact work.

@aviettran I modified it so it works without package-lock.json.  Thank you for the comments

@aviettran I modified it so it works without package-lock.json.  Thank you for the comments

@aviettran yeah don't know how i missed the dest path.  Thanks

@aviettran yeah don't know how i missed the dest path.  Thanks

docker functionality working as expected

docker functionality working as expected

@aviettran which docerfile would you prefer? Sorry none of the members I have asked have used dockerfiles before

The Dockerfiles are very similar and only differ in how they serve http (nginx vs node/express). The API PR adds express routes (an API), however, so that should be the basis by which you decide which PR is more appropriate.


refactor: refactored orderby.ts and formatted orderby.ts and where.ts

refactor: refactored orderby.ts and formatted orderby.ts and where.ts

#12


fix: display verb names

aaron@Aarons-Air jr % cargo run
   Compiling jr v0.1.0 (/Users/aaron/codebases/jr)
    Finished dev [unoptimized + debuginfo] target(s) in 1.50s
     Running `target/debug/jr`
jr 0.1.0
   f=: 3 : 0
NB. comment
2 * y
)
(3 : '
 2 * y
 ')
   f
(3 : '
 2 * y
 ')

Still some remaining display issues:
aaron@Aarons-Air jr % cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.05s
     Running `target/debug/jr`
jr 0.1.0
   2 : 0
u/y
)
unrepresentable conjunction
   /
/
   1 : 0
u/y
)
unrepresentable adverb

I'm not sure what is going on here yet:
   {{ 4 $ 1 2 3 }}
(3 : '4 $ 3 $ 1 2 3 ')

Where is the $ 3 coming from in the rendered verb?

I'm not sure what is going on here yet:
   {{ 4 $ 1 2 3 }}
(3 : '4 $ 3 $ 1 2 3 ')

Where is the $ 3 coming from in the rendered verb?

It was quote_arr() rendering like this.  I've "fixed" this but I'm not sure what the intended use case of quote_arr() was originally.


Bookmark cell dynamic size and fix swipe to delete crash

Closes #812
Guess #720 causes some issues with bookmark cell dynamic sizes. This will fix that and a bonus point it will fix crash on swipe to delete and a custom bookmark cell.

@heshamsalman didn't see that coming! checking right now and will make changes

IOS 11 has default UITableViewAutomaticDimension so self-sizing is enabled by default. However, I'm facing issues on iOS 10 for UITableViewCellStyle as subTitle. Setting UITableViewAutomaticDimension is not actually working.

So default table view cell details label don't have self-sizing on < iOS 11. Since we support from iOS 10. I went ahead and created custom cell + layout. Tested on both iOS 11 and iOS 10 and it just works.

@heshamsalman @rizwankce any concerns w/ how much bookmarks is deviating away from the way we build things in the rest of the app? This requires a complete 180 context shift to work on now 😕
If we were using more specific UITableView features I wouldn't mind, but we aren't. e.g. swiping is done w/ SwipeCellKit, and we're not using editing mode...
There's also the fact that we could just reuse PR/issue and repo cells from other products and standardize things a bit more.
I'll land this as-is, just calling it out. Forwarning that I'm probably going to 100% rewrite this in the future 😛

@rnystrom yeah. Bit weird to go from IGLK mode to table mode. I actually have a branch with a re-write in IGLK but it's definitely gonna need a rebase.
The big thing that bugs me -- more than the context switch -- is pulling in the hack from SwipeCell to get it to work on tables in iOS 11 as well as the duplication around it.
Regards,
Hesham
On Nov 4, 2017, 10:58 AM -0400, Ryan Nystrom<EMAIL_ADDRESS>wrote:

Merged #819.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@heshamsalman that's a great point

Couple followups re: sizing:

min height needs to be larger
v-alignment looks off (label should be vertically centered)

When we started bookmarks. It was simple table view cell with editing mode enabled. But down the line with self sizing and swipe to delete we kind of diviated a lot. I totally agree we should use Iglk. Since there are many ways we can reuse things within our code base.
Sent with GitHawk