Merge remote-tracking branch 'origin/develop' into explicit-addressing

This commit is contained in:
lain 2019-06-04 11:23:46 +02:00
commit 4cab98b7bf
23 changed files with 344 additions and 40 deletions

View file

@ -5,6 +5,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
## [unreleased]
### Added
- Add a generic settings store for frontends / clients to use.
- Explicit addressing option for posting.
- Optional SSH access mode. (Needs `erlang-ssh` package on some distributions).
- [MongooseIM](https://github.com/esl/MongooseIM) http authentication support.
@ -49,6 +50,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- OAuth: added job to clean expired access tokens
- MRF: Support for rejecting reports from specific instances (`mrf_simple`)
- MRF: Support for stripping avatars and banner images from specific instances (`mrf_simple`)
- MRF: Support for running subchains.
### Changed
- **Breaking:** Configuration: move from Pleroma.Mailer to Pleroma.Emails.Mailer

View file

@ -326,6 +326,8 @@ config :pleroma, :mrf_keyword,
federated_timeline_removal: [],
replace: []
config :pleroma, :mrf_subchain, match_actor: %{}
config :pleroma, :rich_media, enabled: true
config :pleroma, :media_proxy,
@ -459,7 +461,11 @@ config :pleroma, :ldap,
config :esshd,
enabled: false
oauth_consumer_strategies = String.split(System.get_env("OAUTH_CONSUMER_STRATEGIES") || "")
oauth_consumer_strategies =
System.get_env("OAUTH_CONSUMER_STRATEGIES")
|> to_string()
|> String.split()
|> Enum.map(&hd(String.split(&1, ":")))
ueberauth_providers =
for strategy <- oauth_consumer_strategies do

View file

@ -43,6 +43,7 @@ Has these additional fields under the `pleroma` object:
- `confirmation_pending`: boolean, true if a new user account is waiting on email confirmation to be activated
- `hide_followers`: boolean, true when the user has follower hiding enabled
- `hide_follows`: boolean, true when the user has follow hiding enabled
- `settings_store`: A generic map of settings for frontends. Opaque to the backend. Only returned in `verify_credentials` and `update_credentials`
### Source
@ -81,6 +82,14 @@ Additional parameters can be added to the JSON body/Form data:
- `hide_favorites` - if true, user's favorites timeline will be hidden
- `show_role` - if true, user's role (e.g admin, moderator) will be exposed to anyone in the API
- `default_scope` - the scope returned under `privacy` key in Source subentity
- `pleroma_settings_store` - Opaque user settings to be saved on the backend.
### Pleroma Settings Store
Pleroma has mechanism that allows frontends to save blobs of json for each user on the backend. This can be used to save frontend-specific settings for a user that the backend does not need to know about.
The parameter should have a form of `{frontend_name: {...}}`, with `frontend_name` identifying your type of client, e.g. `pleroma_fe`. It will overwrite everything under this property, but will not overwrite other frontend's settings.
This information is returned in the `verify_credentials` endpoint.
## Authentication

View file

@ -86,6 +86,7 @@ config :pleroma, Pleroma.Emails.Mailer,
* `Pleroma.Web.ActivityPub.MRF.NoOpPolicy`: Doesnt modify activities (default)
* `Pleroma.Web.ActivityPub.MRF.DropPolicy`: Drops all activities. It generally doesnt makes sense to use in production
* `Pleroma.Web.ActivityPub.MRF.SimplePolicy`: Restrict the visibility of activities from certains instances (See ``:mrf_simple`` section)
* `Pleroma.Web.ActivityPub.MRF.SubchainPolicy`: Selectively runs other MRF policies when messages match (see ``:mrf_subchain`` section)
* `Pleroma.Web.ActivityPub.MRF.RejectNonPublic`: Drops posts with non-public visibility settings (See ``:mrf_rejectnonpublic`` section)
* `Pleroma.Web.ActivityPub.MRF.EnsureRePrepended`: Rewrites posts to ensure that replies to posts with subjects do not have an identical subject and instead begin with re:.
* `public`: Makes the client API in authentificated mode-only except for user-profiles. Useful for disabling the Local Timeline and The Whole Known Network.
@ -229,6 +230,21 @@ relates to mascots on the mastodon frontend
* `avatar_removal`: List of instances to strip avatars from
* `banner_removal`: List of instances to strip banners from
## :mrf_subchain
This policy processes messages through an alternate pipeline when a given message matches certain criteria.
All criteria are configured as a map of regular expressions to lists of policy modules.
* `match_actor`: Matches a series of regular expressions against the actor field.
Example:
```
config :pleroma, :mrf_subchain,
match_actor: %{
~r/https:\/\/example.com/s => [Pleroma.Web.ActivityPub.MRF.DropPolicy]
}
```
## :mrf_rejectnonpublic
* `allow_followersonly`: whether to allow followers-only posts
* `allow_direct`: whether to allow direct messages
@ -497,7 +513,7 @@ Authentication / authorization settings.
* `auth_template`: authentication form template. By default it's `show.html` which corresponds to `lib/pleroma/web/templates/o_auth/o_auth/show.html.eex`.
* `oauth_consumer_template`: OAuth consumer mode authentication form template. By default it's `consumer.html` which corresponds to `lib/pleroma/web/templates/o_auth/o_auth/consumer.html.eex`.
* `oauth_consumer_strategies`: the list of enabled OAuth consumer strategies; by default it's set by OAUTH_CONSUMER_STRATEGIES environment variable.
* `oauth_consumer_strategies`: the list of enabled OAuth consumer strategies; by default it's set by OAUTH_CONSUMER_STRATEGIES environment variable. Each entry in this space-delimited string should be of format `<strategy>` or `<strategy>:<dependency>` (e.g. `twitter` or `keycloak:ueberauth_keycloak_strategy` in case dependency is named differently than `ueberauth_<strategy>`).
## OAuth consumer mode

View file

@ -44,6 +44,7 @@ defmodule Pleroma.User.Info do
field(:pinned_activities, {:array, :string}, default: [])
field(:mascot, :map, default: nil)
field(:emoji, {:array, :map}, default: [])
field(:pleroma_settings_store, :map, default: %{})
field(:notification_settings, :map,
default: %{
@ -218,7 +219,8 @@ defmodule Pleroma.User.Info do
:hide_followers,
:hide_favorites,
:background,
:show_role
:show_role,
:pleroma_settings_store
])
end

View file

@ -5,8 +5,8 @@
defmodule Pleroma.Web.ActivityPub.MRF do
@callback filter(Map.t()) :: {:ok | :reject, Map.t()}
def filter(object) do
get_policies()
def filter(policies, %{} = object) do
policies
|> Enum.reduce({:ok, object}, fn
policy, {:ok, object} ->
policy.filter(object)
@ -16,6 +16,8 @@ defmodule Pleroma.Web.ActivityPub.MRF do
end)
end
def filter(%{} = object), do: get_policies() |> filter(object)
def get_policies do
Pleroma.Config.get([:instance, :rewrite_policy], []) |> get_policies()
end

View file

@ -0,0 +1,40 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Web.ActivityPub.MRF.SubchainPolicy do
alias Pleroma.Config
alias Pleroma.Web.ActivityPub.MRF
require Logger
@behaviour MRF
defp lookup_subchain(actor) do
with matches <- Config.get([:mrf_subchain, :match_actor]),
{match, subchain} <- Enum.find(matches, fn {k, _v} -> String.match?(actor, k) end) do
{:ok, match, subchain}
else
_e -> {:error, :notfound}
end
end
@impl true
def filter(%{"actor" => actor} = message) do
with {:ok, match, subchain} <- lookup_subchain(actor) do
Logger.debug(
"[SubchainPolicy] Matched #{actor} against #{inspect(match)} with subchain #{
inspect(subchain)
}"
)
subchain
|> MRF.filter(message)
else
_e -> {:ok, message}
end
end
@impl true
def filter(message), do: {:ok, message}
end

View file

@ -797,7 +797,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
where: fragment("(?)->>'actor' = ?", activity.data, ^actor),
where:
fragment(
"(?)->'inReplyTo' = ?",
"(?)->>'inReplyTo' = ?",
object.data,
^to_string(id)
),

View file

@ -124,6 +124,9 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
end)
end)
|> add_if_present(params, "default_scope", :default_scope)
|> add_if_present(params, "pleroma_settings_store", :pleroma_settings_store, fn value ->
{:ok, Map.merge(user.info.pleroma_settings_store, value)}
end)
|> add_if_present(params, "header", :banner, fn value ->
with %Plug.Upload{} <- value,
{:ok, object} <- ActivityPub.upload(value, type: :banner) do
@ -143,7 +146,10 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
CommonAPI.update(user)
end
json(conn, AccountView.render("account.json", %{user: user, for: user}))
json(
conn,
AccountView.render("account.json", %{user: user, for: user, with_pleroma_settings: true})
)
else
_e ->
conn
@ -153,7 +159,9 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
end
def verify_credentials(%{assigns: %{user: user}} = conn, _) do
account = AccountView.render("account.json", %{user: user, for: user})
account =
AccountView.render("account.json", %{user: user, for: user, with_pleroma_settings: true})
json(conn, account)
end

View file

@ -130,6 +130,7 @@ defmodule Pleroma.Web.MastodonAPI.AccountView do
|> maybe_put_role(user, opts[:for])
|> maybe_put_settings(user, opts[:for], user_info)
|> maybe_put_notification_settings(user, opts[:for])
|> maybe_put_settings_store(user, opts[:for], opts)
end
defp username_from_nickname(string) when is_binary(string) do
@ -152,6 +153,15 @@ defmodule Pleroma.Web.MastodonAPI.AccountView do
defp maybe_put_settings(data, _, _, _), do: data
defp maybe_put_settings_store(data, %User{info: info, id: id}, %User{id: id}, %{
with_pleroma_settings: true
}) do
data
|> Kernel.put_in([:pleroma, :settings_store], info.pleroma_settings_store)
end
defp maybe_put_settings_store(data, _, _, _), do: data
defp maybe_put_role(data, %User{info: %{show_role: true}} = user, _) do
data
|> Kernel.put_in([:pleroma, :is_admin], user.info.is_admin)

View file

@ -97,6 +97,7 @@ defmodule Pleroma.Web.Nodeinfo.NodeinfoController do
"pleroma_api",
"mastodon_api",
"mastodon_api_streaming",
"polls",
if Config.get([:media_proxy, :enabled]) do
"media_proxy"
end,
@ -149,6 +150,7 @@ defmodule Pleroma.Web.Nodeinfo.NodeinfoController do
},
staffAccounts: staff_accounts,
federation: federation_response,
pollLimits: Config.get([:instance, :poll_limits]),
postFormats: Config.get([:instance, :allowed_post_formats]),
uploadLimits: %{
general: Config.get([:instance, :upload_limit]),

View file

@ -63,13 +63,14 @@
.scopes-input {
display: flex;
flex-direction: column;
margin-top: 1em;
text-align: left;
color: #89898a;
}
.scopes-input label:first-child {
flex-basis: 40%;
height: 2em;
}
.scopes {
@ -80,13 +81,22 @@
}
.scope {
flex-basis: 100%;
display: flex;
flex-basis: 100%;
height: 2em;
align-items: center;
}
.scope:before {
color: #b9b9ba;
content: "\fe0e";
margin-left: 1em;
margin-right: 1em;
}
[type="checkbox"] + label {
display: none;
cursor: pointer;
margin: 0.5em;
}
@ -95,10 +105,12 @@
}
[type="checkbox"] + label:before {
cursor: pointer;
display: inline-block;
color: white;
background-color: #121a24;
border: 4px solid #121a24;
box-shadow: 0px 0px 1px 0 #d8a070;
box-sizing: border-box;
width: 1.2em;
height: 1.2em;
@ -128,7 +140,8 @@
border-radius: 4px;
border: none;
padding: 10px;
margin-top: 30px;
margin-top: 20px;
margin-bottom: 20px;
text-transform: uppercase;
font-size: 16px;
box-shadow: 0px 0px 2px 0px black,
@ -147,8 +160,8 @@
box-sizing: border-box;
width: 100%;
background-color: #931014;
border: 1px solid #a06060;
border-radius: 4px;
border: none;
padding: 10px;
margin-top: 20px;
font-weight: 500;
@ -171,12 +184,27 @@
margin-top: 0
}
.scopes-input {
flex-direction: column;
.scope {
flex-basis: 0%;
}
.scope {
flex-basis: 50%;
.scope:before {
content: "";
margin-left: 0em;
margin-right: 1em;
}
.scope:first-child:before {
margin-left: 1em;
content: "\fe0e";
}
.scope:after {
content: ",";
}
.scope:last-child:after {
content: "";
}
}
.form-row {

View file

@ -1,13 +1,19 @@
<div class="scopes-input">
<%= label @form, :scope, "Permissions" %>
<%= label @form, :scope, "The following permissions will be granted" %>
<div class="scopes">
<%= for scope <- @available_scopes do %>
<%# Note: using hidden input with `unchecked_value` in order to distinguish user's empty selection from `scope` param being omitted %>
<div class="scope">
<%= if scope in @scopes do %>
<div class="scope">
<%= checkbox @form, :"scope_#{scope}", value: scope in @scopes && scope, checked_value: scope, unchecked_value: "", name: "authorization[scope][]" %>
<%= label @form, :"scope_#{scope}", String.capitalize(scope) %>
<%= if scope in @scopes && scope do %>
<%= String.capitalize(scope) %>
<% end %>
</div>
<% else %>
<%= checkbox @form, :"scope_#{scope}", value: scope in @scopes && scope, checked_value: scope, unchecked_value: "", name: "authorization[scope][]" %>
<%= label @form, :"scope_#{scope}", String.capitalize(scope) %>
</div>
<% end %>
<% end %>
</div>
</div>

View file

@ -1,7 +1,9 @@
<h2>Sign in with external provider</h2>
<%= form_for @conn, o_auth_path(@conn, :prepare_request), [as: "authorization", method: "get"], fn f -> %>
<%= render @view_module, "_scopes.html", Map.put(assigns, :form, f) %>
<div style="display: none">
<%= render @view_module, "_scopes.html", Map.merge(assigns, %{form: f}) %>
</div>
<%= hidden_input f, :client_id, value: @client_id %>
<%= hidden_input f, :redirect_uri, value: @redirect_uri %>

View file

@ -6,26 +6,38 @@
<% end %>
<h2>OAuth Authorization</h2>
<%= form_for @conn, o_auth_path(@conn, :authorize), [as: "authorization"], fn f -> %>
<div class="input">
<%= label f, :name, "Name or email" %>
<%= text_input f, :name %>
</div>
<div class="input">
<%= label f, :password, "Password" %>
<%= password_input f, :password %>
</div>
<%= render @view_module, "_scopes.html", Map.merge(assigns, %{form: f}) %>
<%= if @params["registration"] in ["true", true] do %>
<h3>This is the first time you visit! Please enter your Pleroma handle.</h3>
<p>Choose carefully! You won't be able to change this later. You will be able to change your display name, though.</p>
<div class="input">
<%= label f, :nickname, "Pleroma Handle" %>
<%= text_input f, :nickname, placeholder: "lain" %>
</div>
<%= hidden_input f, :name, value: @params["name"] %>
<%= hidden_input f, :password, value: @params["password"] %>
<br>
<% else %>
<div class="input">
<%= label f, :name, "Username" %>
<%= text_input f, :name %>
</div>
<div class="input">
<%= label f, :password, "Password" %>
<%= password_input f, :password %>
</div>
<%= submit "Log In" %>
<%= render @view_module, "_scopes.html", Map.merge(assigns, %{form: f}) %>
<% end %>
<%= hidden_input f, :client_id, value: @client_id %>
<%= hidden_input f, :response_type, value: @response_type %>
<%= hidden_input f, :redirect_uri, value: @redirect_uri %>
<%= hidden_input f, :state, value: @state %>
<%= submit "Authorize" %>
<% end %>
<%= if Pleroma.Config.oauth_consumer_enabled?() do %>
<%= render @view_module, Pleroma.Web.Auth.Authenticator.oauth_consumer_template(), assigns %>
<% end %>

25
mix.exs
View file

@ -51,16 +51,27 @@ defmodule Pleroma.Mixfile do
defp elixirc_paths(:test), do: ["lib", "test/support"]
defp elixirc_paths(_), do: ["lib"]
# Specifies OAuth dependencies.
defp oauth_deps do
oauth_strategy_packages =
System.get_env("OAUTH_CONSUMER_STRATEGIES")
|> to_string()
|> String.split()
|> Enum.map(fn strategy_entry ->
with [_strategy, dependency] <- String.split(strategy_entry, ":") do
dependency
else
[strategy] -> "ueberauth_#{strategy}"
end
end)
for s <- oauth_strategy_packages, do: {String.to_atom(s), ">= 0.0.0"}
end
# Specifies your project dependencies.
#
# Type `mix help deps` for examples and options.
defp deps do
oauth_strategies = String.split(System.get_env("OAUTH_CONSUMER_STRATEGIES") || "")
oauth_deps =
for s <- oauth_strategies,
do: {String.to_atom("ueberauth_#{s}"), ">= 0.0.0"}
[
{:phoenix, "~> 1.4.1"},
{:plug_cowboy, "~> 2.0"},
@ -121,7 +132,7 @@ defmodule Pleroma.Mixfile do
{:ex_rated, "~> 1.2"},
{:plug_static_index_html, "~> 1.0.0"},
{:excoveralls, "~> 0.11.1", only: :test}
] ++ oauth_deps
] ++ oauth_deps()
end
# Aliases are shortcuts or tasks specific to the current project.

View file

@ -0,0 +1,7 @@
defmodule Pleroma.Repo.Migrations.AddObjectInReplyToIndex do
use Ecto.Migration
def change do
create index(:objects, ["(data->>'inReplyTo')"], name: :objects_in_reply_to_index)
end
end

View file

@ -0,0 +1,8 @@
defmodule Pleroma.Repo.Migrations.AddTagIndexToObjects do
use Ecto.Migration
def change do
drop_if_exists index(:activities, ["(data #> '{\"object\",\"tag\"}')"], using: :gin, name: :activities_tags)
create index(:objects, ["(data->'tag')"], using: :gin, name: :objects_tags)
end
end

View file

@ -6,6 +6,11 @@ defmodule Pleroma.Object.ContainmentTest do
import Pleroma.Factory
setup_all do
Tesla.Mock.mock_global(fn env -> apply(HttpRequestMock, :request, [env]) end)
:ok
end
describe "general origin containment" do
test "contain_origin_from_id() catches obvious spoofing attempts" do
data = %{

View file

@ -243,6 +243,14 @@ defmodule HttpRequestMock do
}}
end
def get("https://n1u.moe/users/rye", _, _, Accept: "application/activity+json") do
{:ok,
%Tesla.Env{
status: 200,
body: File.read!("test/fixtures/httpoison_mock/rye.json")
}}
end
def get("http://mastodon.example.org/users/admin/statuses/100787282858396771", _, _, _) do
{:ok,
%Tesla.Env{
@ -302,6 +310,10 @@ defmodule HttpRequestMock do
}}
end
def get("http://mastodon.example.org/users/gargron", _, _, Accept: "application/activity+json") do
{:error, :nxdomain}
end
def get(
"http://mastodon.example.org/@admin/99541947525187367",
_,
@ -546,6 +558,15 @@ defmodule HttpRequestMock do
}}
end
def get(
"http://gs.example.org:4040/index.php/user/1",
_,
_,
Accept: "application/activity+json"
) do
{:ok, %Tesla.Env{status: 406, body: ""}}
end
def get("http://gs.example.org/index.php/api/statuses/user_timeline/1.atom", _, _, _) do
{:ok,
%Tesla.Env{

View file

@ -0,0 +1,32 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Web.ActivityPub.MRF.SubchainPolicyTest do
use Pleroma.DataCase
alias Pleroma.Web.ActivityPub.MRF.DropPolicy
alias Pleroma.Web.ActivityPub.MRF.SubchainPolicy
@message %{
"actor" => "https://banned.com",
"type" => "Create",
"object" => %{"content" => "hi"}
}
test "it matches and processes subchains when the actor matches a configured target" do
Pleroma.Config.put([:mrf_subchain, :match_actor], %{
~r/^https:\/\/banned.com/s => [DropPolicy]
})
{:reject, _} = SubchainPolicy.filter(@message)
end
test "it doesn't match and process subchains when the actor doesn't match a configured target" do
Pleroma.Config.put([:mrf_subchain, :match_actor], %{
~r/^https:\/\/borked.com/s => [DropPolicy]
})
{:ok, _message} = SubchainPolicy.filter(@message)
end
end

View file

@ -239,4 +239,19 @@ defmodule Pleroma.Web.MastodonAPI.AccountViewTest do
assert expected == AccountView.render("account.json", %{user: user, for: other_user})
end
test "returns the settings store if the requesting user is the represented user and it's requested specifically" do
user = insert(:user, %{info: %User.Info{pleroma_settings_store: %{fe: "test"}}})
result =
AccountView.render("account.json", %{user: user, for: user, with_pleroma_settings: true})
assert result.pleroma.settings_store == %{:fe => "test"}
result = AccountView.render("account.json", %{user: user, with_pleroma_settings: true})
assert result.pleroma[:settings_store] == nil
result = AccountView.render("account.json", %{user: user, for: user})
assert result.pleroma[:settings_store] == nil
end
end

View file

@ -2423,6 +2423,66 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIControllerTest do
end
describe "updating credentials" do
test "sets user settings in a generic way", %{conn: conn} do
user = insert(:user)
res_conn =
conn
|> assign(:user, user)
|> patch("/api/v1/accounts/update_credentials", %{
"pleroma_settings_store" => %{
pleroma_fe: %{
theme: "bla"
}
}
})
assert user = json_response(res_conn, 200)
assert user["pleroma"]["settings_store"] == %{"pleroma_fe" => %{"theme" => "bla"}}
user = Repo.get(User, user["id"])
res_conn =
conn
|> assign(:user, user)
|> patch("/api/v1/accounts/update_credentials", %{
"pleroma_settings_store" => %{
masto_fe: %{
theme: "bla"
}
}
})
assert user = json_response(res_conn, 200)
assert user["pleroma"]["settings_store"] ==
%{
"pleroma_fe" => %{"theme" => "bla"},
"masto_fe" => %{"theme" => "bla"}
}
user = Repo.get(User, user["id"])
res_conn =
conn
|> assign(:user, user)
|> patch("/api/v1/accounts/update_credentials", %{
"pleroma_settings_store" => %{
masto_fe: %{
theme: "blub"
}
}
})
assert user = json_response(res_conn, 200)
assert user["pleroma"]["settings_store"] ==
%{
"pleroma_fe" => %{"theme" => "bla"},
"masto_fe" => %{"theme" => "blub"}
}
end
test "updates the user's bio", %{conn: conn} do
user = insert(:user)
user2 = insert(:user)