2db9a4275c
TL;DR - GDB can hang if something refreshes the thread list out of the target while the target is running. GDB hangs inside td_ta_thr_iter. The fix is to not use that libthread_db function anymore. Long version: Running the testsuite against my all-stop-on-top-of-non-stop series is still exposing latent non-stop bugs. I was originally seeing this with the multi-create.exp test, back when we were still using libthread_db thread event breakpoints. The all-stop-on-top-of-non-stop series forces a thread list refresh each time GDB needs to start stepping over a breakpoint (to pause all threads). That test hits the thread event breakpoint often, resulting in a bunch of step-over operations, thus a bunch of thread list refreshes while some threads in the target are running. The commit adds a real non-stop mode test that triggers the issue, based on multi-create.exp, that does an explicit "info threads" when a breakpoint is hit. IOW, it does the same things the as-ns series was doing when testing multi-create.exp. The bug is a race, so it unfortunately takes several runs for the test to trigger it. In fact, even when setting the test running in a loop, it sometimes takes several minutes for it to trigger for me. The race is related to libthread_db's td_ta_thr_iter. This is libthread_db's entry point for walking the thread list of the inferior. Sometimes, when GDB refreshes the thread list from the target, libthread_db's td_ta_thr_iter can somehow see glibc's thread list as a cycle, and get stuck in an infinite loop. The issue is that when a thread exits, its thread control structure in glibc is moved from a "used" list to a "cache" list. These lists are simply circular linked lists where the "next/prev" pointers are embedded in the thread control structure itself. The "next" pointer of the last element of the list points back to the list's sentinel "head". There's only one set of "next/prev" pointers for both lists; thus a thread can only be in one of the lists at a time, not in both simultaneously. So when thread C exits, simplifying, the following happens. A-C are threads. stack_used and stack_cache are the list's heads. Before: stack_used -> A -> B -> C -> (&stack_used) stack_cache -> (&stack_cache) After: stack_used -> A -> B -> (&stack_used) stack_cache -> C -> (&stack_cache) td_ta_thr_iter starts by iterating at the list's head's next, and iterates until it sees a thread whose next pointer points to the list's head again. Thus in the before case above, C's next points to stack_used, indicating end of list. In the same case, the stack_cache list is empty. For each thread being iterated, td_ta_thr_iter reads the whole thread object out of the inferior. This includes the thread's "next" pointer. In the scenario above, it may happen that td_ta_thr_iter is iterating thread B and has already read B's thread structure just before thread C exits and its control structure moves to the cached list. Now, recall that td_ta_thr_iter is running in the context of GDB, and there's no locking between GDB and the inferior. From it's local copy of B, td_ta_thr_iter believes that the next thread after B is thread C, so it happilly continues iterating to C, a thread that has already exited, and is now in the stack cache list. After iterating C, td_ta_thr_iter finds the stack_cache head, which because it is not stack_used, td_ta_thr_iter assumes it's just another thread. After this, unless the reverse race triggers, GDB gets stuck in td_ta_thr_iter forever walking the stack_cache list, as no thread in thatlist has a next pointer that points back to stack_used (the terminating condition). Before fully understanding the issue, I tried adding cycle detection to GDB's td_ta_thr_iter callback. However, td_ta_thr_iter skips calling the callback in some cases, which means that it's possible that the callback isn't called at all, making it impossible for GDB to break the loop. I did manage to get GDB stuck in that state more than once. Fortunately, we can avoid the issue altogether. We don't really need td_ta_thr_iter for live debugging nowadays, given PTRACE_EVENT_CLONE. We already know how to map and lwp id to a thread id without iterating (thread_from_lwp), so use that more. gdb/ChangeLog: 2015-02-20 Pedro Alves <palves@redhat.com> * linux-nat.c (linux_handle_extended_wait): Call thread_db_notice_clone whenever a new clone LWP is detected. (linux_stop_and_wait_all_lwps, linux_unstop_all_lwps): New functions. * linux-nat.h (thread_db_attach_lwp): Delete declaration. (thread_db_notice_clone, linux_stop_and_wait_all_lwps) (linux_unstop_all_lwps): Declare. * linux-thread-db.c (struct thread_get_info_inout): Delete. (thread_get_info_callback): Delete. (thread_from_lwp): Use td_thr_get_info and record_thread. (thread_db_attach_lwp): Delete. (thread_db_notice_clone): New function. (try_thread_db_load_1): If /proc is mounted and shows the process'es task list, walk over all LWPs and call thread_from_lwp instead of relying on td_ta_thr_iter. (attach_thread): Don't call check_thread_signals here. Split the tail part of the function (which adds the thread to the core GDB thread list) to ... (record_thread): ... this function. Call check_thread_signals here. (thread_db_wait): Don't call thread_db_find_new_threads_1. Always call thread_from_lwp. (thread_db_update_thread_list): Rename to ... (thread_db_update_thread_list_org): ... this. (thread_db_update_thread_list): New function. (thread_db_find_thread_from_tid): Delete. (thread_db_get_ada_task_ptid): Simplify. * nat/linux-procfs.c: Include <sys/stat.h>. (linux_proc_task_list_dir_exists): New function. * nat/linux-procfs.h (linux_proc_task_list_dir_exists): Declare. gdb/gdbserver/ChangeLog: 2015-02-20 Pedro Alves <palves@redhat.com> * thread-db.c: Include "nat/linux-procfs.h". (thread_db_init): Skip listing new threads if the kernel supports PTRACE_EVENT_CLONE and /proc/PID/task/ is accessible. gdb/testsuite/ChangeLog: 2015-02-20 Pedro Alves <palves@redhat.com> * gdb.threads/multi-create-ns-info-thr.exp: New file.
266 lines
6.1 KiB
C
266 lines
6.1 KiB
C
/* Linux-specific PROCFS manipulation routines.
|
|
Copyright (C) 2009-2015 Free Software Foundation, Inc.
|
|
|
|
This file is part of GDB.
|
|
|
|
This program is free software; you can redistribute it and/or modify
|
|
it under the terms of the GNU General Public License as published by
|
|
the Free Software Foundation; either version 3 of the License, or
|
|
(at your option) any later version.
|
|
|
|
This program is distributed in the hope that it will be useful,
|
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
GNU General Public License for more details.
|
|
|
|
You should have received a copy of the GNU General Public License
|
|
along with this program. If not, see <http://www.gnu.org/licenses/>. */
|
|
|
|
#include "common-defs.h"
|
|
#include "linux-procfs.h"
|
|
#include "filestuff.h"
|
|
#include <dirent.h>
|
|
#include <sys/stat.h>
|
|
|
|
/* Return the TGID of LWPID from /proc/pid/status. Returns -1 if not
|
|
found. */
|
|
|
|
static int
|
|
linux_proc_get_int (pid_t lwpid, const char *field, int warn)
|
|
{
|
|
size_t field_len = strlen (field);
|
|
FILE *status_file;
|
|
char buf[100];
|
|
int retval = -1;
|
|
|
|
snprintf (buf, sizeof (buf), "/proc/%d/status", (int) lwpid);
|
|
status_file = gdb_fopen_cloexec (buf, "r");
|
|
if (status_file == NULL)
|
|
{
|
|
if (warn)
|
|
warning (_("unable to open /proc file '%s'"), buf);
|
|
return -1;
|
|
}
|
|
|
|
while (fgets (buf, sizeof (buf), status_file))
|
|
if (strncmp (buf, field, field_len) == 0 && buf[field_len] == ':')
|
|
{
|
|
retval = strtol (&buf[field_len + 1], NULL, 10);
|
|
break;
|
|
}
|
|
|
|
fclose (status_file);
|
|
return retval;
|
|
}
|
|
|
|
/* Return the TGID of LWPID from /proc/pid/status. Returns -1 if not
|
|
found. */
|
|
|
|
int
|
|
linux_proc_get_tgid (pid_t lwpid)
|
|
{
|
|
return linux_proc_get_int (lwpid, "Tgid", 1);
|
|
}
|
|
|
|
/* See linux-procfs.h. */
|
|
|
|
pid_t
|
|
linux_proc_get_tracerpid_nowarn (pid_t lwpid)
|
|
{
|
|
return linux_proc_get_int (lwpid, "TracerPid", 0);
|
|
}
|
|
|
|
/* Fill in BUFFER, a buffer with BUFFER_SIZE bytes with the 'State'
|
|
line of /proc/PID/status. Returns -1 on failure to open the /proc
|
|
file, 1 if the line is found, and 0 if not found. If WARN, warn on
|
|
failure to open the /proc file. */
|
|
|
|
static int
|
|
linux_proc_pid_get_state (pid_t pid, char *buffer, size_t buffer_size,
|
|
int warn)
|
|
{
|
|
FILE *procfile;
|
|
int have_state;
|
|
|
|
xsnprintf (buffer, buffer_size, "/proc/%d/status", (int) pid);
|
|
procfile = gdb_fopen_cloexec (buffer, "r");
|
|
if (procfile == NULL)
|
|
{
|
|
if (warn)
|
|
warning (_("unable to open /proc file '%s'"), buffer);
|
|
return -1;
|
|
}
|
|
|
|
have_state = 0;
|
|
while (fgets (buffer, buffer_size, procfile) != NULL)
|
|
if (strncmp (buffer, "State:", 6) == 0)
|
|
{
|
|
have_state = 1;
|
|
break;
|
|
}
|
|
fclose (procfile);
|
|
return have_state;
|
|
}
|
|
|
|
/* See linux-procfs.h declaration. */
|
|
|
|
int
|
|
linux_proc_pid_is_gone (pid_t pid)
|
|
{
|
|
char buffer[100];
|
|
int have_state;
|
|
|
|
have_state = linux_proc_pid_get_state (pid, buffer, sizeof buffer, 0);
|
|
if (have_state < 0)
|
|
{
|
|
/* If we can't open the status file, assume the thread has
|
|
disappeared. */
|
|
return 1;
|
|
}
|
|
else if (have_state == 0)
|
|
{
|
|
/* No "State:" line, assume thread is alive. */
|
|
return 0;
|
|
}
|
|
else
|
|
{
|
|
return (strstr (buffer, "Z (") != NULL
|
|
|| strstr (buffer, "X (") != NULL);
|
|
}
|
|
}
|
|
|
|
/* Return non-zero if 'State' of /proc/PID/status contains STATE. If
|
|
WARN, warn on failure to open the /proc file. */
|
|
|
|
static int
|
|
linux_proc_pid_has_state (pid_t pid, const char *state, int warn)
|
|
{
|
|
char buffer[100];
|
|
int have_state;
|
|
|
|
have_state = linux_proc_pid_get_state (pid, buffer, sizeof buffer, warn);
|
|
return (have_state > 0 && strstr (buffer, state) != NULL);
|
|
}
|
|
|
|
/* Detect `T (stopped)' in `/proc/PID/status'.
|
|
Other states including `T (tracing stop)' are reported as false. */
|
|
|
|
int
|
|
linux_proc_pid_is_stopped (pid_t pid)
|
|
{
|
|
return linux_proc_pid_has_state (pid, "T (stopped)", 1);
|
|
}
|
|
|
|
/* Return non-zero if PID is a zombie. If WARN, warn on failure to
|
|
open the /proc file. */
|
|
|
|
static int
|
|
linux_proc_pid_is_zombie_maybe_warn (pid_t pid, int warn)
|
|
{
|
|
return linux_proc_pid_has_state (pid, "Z (zombie)", warn);
|
|
}
|
|
|
|
/* See linux-procfs.h declaration. */
|
|
|
|
int
|
|
linux_proc_pid_is_zombie_nowarn (pid_t pid)
|
|
{
|
|
return linux_proc_pid_is_zombie_maybe_warn (pid, 0);
|
|
}
|
|
|
|
/* See linux-procfs.h declaration. */
|
|
|
|
int
|
|
linux_proc_pid_is_zombie (pid_t pid)
|
|
{
|
|
return linux_proc_pid_is_zombie_maybe_warn (pid, 1);
|
|
}
|
|
|
|
/* See linux-procfs.h declaration. */
|
|
|
|
char *
|
|
linux_proc_pid_get_ns (pid_t pid, const char *ns)
|
|
{
|
|
char buf[100];
|
|
char nsval[64];
|
|
int ret;
|
|
xsnprintf (buf, sizeof (buf), "/proc/%d/ns/%s", (int) pid, ns);
|
|
ret = readlink (buf, nsval, sizeof (nsval));
|
|
if (0 < ret && ret < sizeof (nsval))
|
|
{
|
|
nsval[ret] = '\0';
|
|
return xstrdup (nsval);
|
|
}
|
|
|
|
return NULL;
|
|
}
|
|
|
|
/* See linux-procfs.h. */
|
|
|
|
void
|
|
linux_proc_attach_tgid_threads (pid_t pid,
|
|
linux_proc_attach_lwp_func attach_lwp)
|
|
{
|
|
DIR *dir;
|
|
char pathname[128];
|
|
int new_threads_found;
|
|
int iterations;
|
|
|
|
if (linux_proc_get_tgid (pid) != pid)
|
|
return;
|
|
|
|
xsnprintf (pathname, sizeof (pathname), "/proc/%ld/task", (long) pid);
|
|
dir = opendir (pathname);
|
|
if (dir == NULL)
|
|
{
|
|
warning (_("Could not open /proc/%ld/task."), (long) pid);
|
|
return;
|
|
}
|
|
|
|
/* Scan the task list for existing threads. While we go through the
|
|
threads, new threads may be spawned. Cycle through the list of
|
|
threads until we have done two iterations without finding new
|
|
threads. */
|
|
for (iterations = 0; iterations < 2; iterations++)
|
|
{
|
|
struct dirent *dp;
|
|
|
|
new_threads_found = 0;
|
|
while ((dp = readdir (dir)) != NULL)
|
|
{
|
|
unsigned long lwp;
|
|
|
|
/* Fetch one lwp. */
|
|
lwp = strtoul (dp->d_name, NULL, 10);
|
|
if (lwp != 0)
|
|
{
|
|
ptid_t ptid = ptid_build (pid, lwp, 0);
|
|
|
|
if (attach_lwp (ptid))
|
|
new_threads_found = 1;
|
|
}
|
|
}
|
|
|
|
if (new_threads_found)
|
|
{
|
|
/* Start over. */
|
|
iterations = -1;
|
|
}
|
|
|
|
rewinddir (dir);
|
|
}
|
|
|
|
closedir (dir);
|
|
}
|
|
|
|
/* See linux-procfs.h. */
|
|
|
|
int
|
|
linux_proc_task_list_dir_exists (pid_t pid)
|
|
{
|
|
char pathname[128];
|
|
struct stat buf;
|
|
|
|
xsnprintf (pathname, sizeof (pathname), "/proc/%ld/task", (long) pid);
|
|
return (stat (pathname, &buf) == 0);
|
|
}
|